vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #272
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.aEp3QP6aut', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: d8c68e5c-7e23-4bf2-a77d-e7ad2822ac6a16(finished: start all VMs, in 72.35 seconds)17gitlab # Superblock backups stored on blocks:18gitlab: waiting for unit setup-gitlab.service19gitlab # 32768, 98304, 163840, 22937620gitlab: waiting for the VM to finish booting21gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hsdm6hk3kxdrv96gkn9jddzas26sjqg4-nixos-system-gitlab-test/init regInfo=/nix/store/v2bcg6f4b41s0vcfz79zw7ccvsaw53nj-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/v2bcg6f4b41s0vcfz79zw7ccvsaw53nj-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000029] arm-pv: using stolen time PV108gitlab # [ 0.000370] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000562] Console: colour dummy device 80x25110gitlab # [ 0.000568] printk: legacy console [tty0] enabled111gitlab # [ 0.000743] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000750] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000827] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000961] landlock: Up and running.115gitlab # [ 0.000964] Yama: becoming mindful.116gitlab # [ 0.001370] LSM support for eBPF active117gitlab # [ 0.001548] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001650] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002799] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003472] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003476] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003654] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004609] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.004701] EFI services will not be available.125gitlab # [ 0.004874] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005617] Detected PIPT I-cache on CPU1127gitlab # [ 0.005717] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005838] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.005963] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007254] Detected PIPT I-cache on CPU2131gitlab # [ 0.007357] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007481] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007605] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008716] Detected PIPT I-cache on CPU3135gitlab # [ 0.008832] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008953] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.009078] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.009634] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.009660] SMP: Total of 4 processors activated.140gitlab # [ 0.009662] CPU: All CPU(s) started at EL1141gitlab # [ 0.009671] CPU features: detected: Branch Target Identification142gitlab # [ 0.009675] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.009678] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.009682] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.009685] CPU features: detected: Common not Private translations146gitlab # [ 0.009688] CPU features: detected: CRC32 instructions147gitlab # [ 0.009690] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.009694] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.009697] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.009700] CPU features: detected: E0PD151gitlab # [ 0.009702] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.009705] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.009708] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.009711] CPU features: detected: Fine Grained Traps155gitlab # [ 0.009715] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.009719] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.009721] CPU features: detected: LSE atomic instructions158gitlab # [ 0.009724] CPU features: detected: Privileged Access Never159gitlab # [ 0.009727] CPU features: detected: PMUv3160gitlab # [ 0.009729] CPU features: detected: RAS Extension Support161gitlab # [ 0.009732] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.009735] CPU features: detected: Random Number Generator163gitlab # [ 0.009737] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.009740] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.009742] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.009746] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.009870] alternatives: applying system-wide alternatives168gitlab # [ 0.012818] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.013014] Memory: 5963804K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.014452] devtmpfs: initialized171gitlab # [ 0.018388] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.018456] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.018646] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.018650] 0 pages in range for non-PLT usage175gitlab # [ 0.018651] 508288 pages in range for PLT usage176gitlab # [ 0.018746] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.019532] DMI not present or invalid.178gitlab # [ 0.022638] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.025255] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.025539] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.027286] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.027308] audit: initializing netlink subsys (disabled)183gitlab # [ 0.027638] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.028120] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.028123] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.028127] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.028130] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.028133] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.028213] cpuidle: using governor ladder190gitlab # [ 0.028238] cpuidle: using governor menu191gitlab # [ 0.028437] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.028643] ASID allocator initialised with 65536 entries193gitlab # [ 0.029803] Serial: AMBA PL011 UART driver194gitlab # [ 0.034917] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.035256] printk: console [ttyAMA0] enabled196gitlab # [ 0.047385] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.047407] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.047418] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.047426] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.047434] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.047442] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.047451] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.047459] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.059022] fbcon: Taking over console205gitlab # [ 0.059033] ACPI: Interpreter disabled.206gitlab # [ 0.060000] iommu: Default domain type: Translated207gitlab # [ 0.060004] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.073136] SCSI subsystem initialized209gitlab # [ 0.074285] usbcore: registered new interface driver usbfs210gitlab # [ 0.074372] usbcore: registered new interface driver hub211gitlab # [ 0.074446] usbcore: registered new device driver usb212gitlab # [ 0.075390] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.075400] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.075423] PTP clock support registered215gitlab # [ 0.075528] EDAC MC: Ver: 3.0.0216gitlab # [ 0.076021] scmi_core: SCMI protocol bus registered217gitlab # [ 0.076901] FPGA manager framework218gitlab # [ 0.079453] vgaarb: loaded219gitlab # [ 0.080245] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.080694] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.080716] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.081795] netfs: FS-Cache loaded223gitlab # [ 0.081987] pnp: PnP ACPI: disabled224gitlab # [ 0.086503] NET: Registered PF_INET protocol family225gitlab # [ 0.087495] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.117881] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.117929] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.117975] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.118266] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.118835] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.118924] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.118975] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.119093] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.119309] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.119326] NET: Registered PF_XDP protocol family236gitlab # [ 0.119340] PCI: CLS 0 bytes, default 64237gitlab # [ 0.119684] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.124937] kvm [1]: HYP mode not available239gitlab # [ 0.168906] Initialise system trusted keyrings240gitlab # [ 0.169331] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.172539] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.176324] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.188360] Key type asymmetric registered244gitlab # [ 0.188365] Asymmetric key parser 'x509' registered245gitlab # [ 0.188426] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.188795] io scheduler mq-deadline registered247gitlab # [ 0.188815] io scheduler kyber registered248gitlab # [ 0.204358] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.208752] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.209162] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.209176] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.209187] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.209193] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.209213] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.209234] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.209312] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.209319] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.209323] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.209327] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.209330] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.209425] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.209971] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.210185] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.210199] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.210243] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.210257] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.210799] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.211018] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.211031] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.211065] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.211601] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.211813] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.211826] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.211855] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.212461] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.212697] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.212709] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.212743] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.213294] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.213507] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.213520] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.213554] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.214112] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.214327] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.214339] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.214372] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.214927] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.215139] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.215152] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.215185] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.215199] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.215775] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.216003] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.216032] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.216618] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.216837] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.216866] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.217328] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.217544] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.217824] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.218116] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.218129] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.218160] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.218700] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.218917] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.218930] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.218964] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.219628] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.219635] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.219639] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.219685] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.219731] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.219780] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.219827] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.219877] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.219924] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.219982] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.220031] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.220083] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.220147] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.220194] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.220220] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.220239] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.220276] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.220301] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.220327] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.220353] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.220377] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.220402] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.220426] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.220451] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.220476] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.220500] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.220525] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.220550] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.220574] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.220594] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.220613] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.220632] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.220652] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.220671] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.220701] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.220706] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.220710] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.222156] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.254473] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.260367] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.267279] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.272901] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.278962] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.285506] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.291655] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.298471] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.304595] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.310666] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.317452] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.340758] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.345304] msm_serial: driver initialized359gitlab # [ 0.345717] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.345889] STM32 USART driver initialized361gitlab # [ 0.394965] loop: module loaded362gitlab # [ 0.395523] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.400491] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.405190] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.410284] virtio_blk virtio5: [vdb] 12237848 512-byte logical blocks (6.27 GB/5.83 GiB)366gitlab # [ 0.415833] megasas: 07.734.00.00-rc1367gitlab # [ 0.417627] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.423017] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.423313] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.427601] Using buffer write method371gitlab # [ 0.427664] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.433828] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.433864] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.437576] Using buffer write method375gitlab # [ 0.437612] Concatenating MTD devices:376gitlab # [ 0.437618] (0): "0.flash"377gitlab # [ 0.437624] (1): "0.flash"378gitlab # [ 0.437629] into device "0.flash"379gitlab # [ 0.571403] Freeing initrd memory: 27184K380gitlab # [ 0.588901] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.597663] thunder_xcv, ver 1.0382gitlab # [ 0.597770] thunder_bgx, ver 1.0383gitlab # [ 0.597842] nicpf, ver 1.0384gitlab # [ 0.599726] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.599740] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.599809] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.599823] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.599891] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.599900] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.599971] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.599982] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.600429] sky2: driver version 1.30393gitlab # [ 0.605256] usbcore: registered new interface driver usb-storage394gitlab # [ 0.605641] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.605638] ehci-pci 0000:00:0a.0: EHCI Host Controller396gitlab # [ 0.605687] usbserial: USB Serial support registered for generic397gitlab # [ 0.605687] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.606475] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.607669] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.610285] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.610344] rtc-pl031 9010000.pl031: setting system clock to 2026-08-28T11:35:03 UTC (1787916903)402gitlab # [ 0.611289] i2c_dev: i2c /dev entries driver403gitlab # [ 0.616361] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00404gitlab # [ 0.617518] hub 1-0:1.0: USB hub found405gitlab # [ 0.617589] hub 1-0:1.0: 6 ports detected406gitlab # [ 0.619907] sdhci: Secure Digital Host Controller Interface driver407gitlab # [ 0.619921] sdhci: Copyright(c) Pierre Ossman408gitlab # [ 0.620806] Synopsys Designware Multimedia Card Interface Driver409gitlab # [ 0.621949] sdhci-pltfm: SDHCI platform and OF driver helper410gitlab # [ 0.626655] hid: raw HID events driver (C) Jiri Kosina411gitlab # [ 0.627505] usbcore: registered new interface driver usbhid412gitlab # [ 0.627515] usbhid: USB HID core driver413gitlab # [ 0.638693] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.643362] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.643635] NET: Registered PF_INET6 protocol family416gitlab # [ 0.646282] Segment Routing with IPv6417gitlab # [ 0.646319] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.646447] NET: Registered PF_PACKET protocol family419gitlab # [ 0.646793] 9pnet: Installing 9P2000 support420gitlab # [ 0.651531] Key type dns_resolver registered421gitlab # [ 0.670387] registered taskstats version 1422gitlab # [ 0.670895] Loading compiled-in X.509 certificates423gitlab # [ 0.698339] Demotion targets for Node 0: null424gitlab # [ 0.698634] Key type .fscrypt registered425gitlab # [ 0.698643] Key type fscrypt-provisioning registered426gitlab # [ 0.698930] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.698956] ima: Allocated hash algorithm: sha1428gitlab # [ 0.699032] ima: No architecture policies found429gitlab # [ 0.700621] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.731612] clk: Disabling unused clocks431gitlab # [ 0.731644] PM: genpd: Disabling unused power domains432gitlab # [ 0.737286] Freeing unused kernel memory: 4736K433gitlab # [ 0.737548] Run /init as init process434gitlab # [ 0.779184] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.864369] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.018986] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.112938] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0438gitlab # [ 1.113924] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)439gitlab # [ 1.113971] systemd[1]: Detected virtualization qemu.440gitlab # [ 1.114034] systemd[1]: Detected architecture arm64.441gitlab # [ 1.114049] systemd[1]: Running in initrd.442gitlab # [ 1.114890] systemd[1]: Initializing machine ID from random generator.443gitlab # [ 1.115154] systemd[1]: Hostname set to <gitlab>.444gitlab # [ 1.232323] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.364101] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.399090] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2447gitlab # [ 1.399531] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0448gitlab # [ 1.477707] systemd[1]: Queued start job for default target Initrd Default Target.449gitlab # [ 1.500805] systemd[1]: Created slice Slice /system/modprobe.450gitlab # [ 1.501165] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.451gitlab # [ 1.501225] systemd[1]: Expecting device /dev/disk/by-label/nix-store...452gitlab # [ 1.501246] systemd[1]: Expecting device /dev/disk/by-label/nixos...453gitlab # [ 1.501281] systemd[1]: Reached target Path Units.454gitlab # [ 1.501310] systemd[1]: Reached target Slice Units.455gitlab # [ 1.501338] systemd[1]: Reached target Swaps.456gitlab # [ 1.501369] systemd[1]: Reached target Timer Units.457gitlab # [ 1.501621] systemd[1]: Listening on D-Bus System Message Bus Socket.458gitlab # [ 1.501880] systemd[1]: Listening on Journal Socket (/dev/log).459gitlab # [ 1.502118] systemd[1]: Listening on Journal Sockets.460gitlab # [ 1.502352] systemd[1]: Listening on udev Control Socket.461gitlab # [ 1.502485] systemd[1]: Listening on udev Kernel Socket.462gitlab # [ 1.502520] systemd[1]: Reached target Socket Units.463gitlab # [ 1.505117] systemd[1]: Starting Create List of Static Device Nodes...464gitlab # [ 1.507585] systemd[1]: Starting Load Kernel Module 9pnet_virtio...465gitlab # [ 1.507685] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs466gitlab # [ 1.510401] systemd[1]: Mounting Kernel Configuration File System...467gitlab # [ 1.536429] systemd[1]: Starting Journal Service...468gitlab # [ 1.543809] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.544170] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.548695] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.555527] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.561333] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.565113] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.567893] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.575402] systemd-journald[94]: Collecting audit messages is disabled.476gitlab # [ 1.578163] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...477gitlab # [ 1.586873] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.478gitlab # [ 1.588447] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev479gitlab # [ 1.595907] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0480gitlab # [ 1.596188] [drm] features: -virgl +edid -resource_blob -host_visible481gitlab # [ 1.596194] [drm] features: -context_init482gitlab # [ 1.597669] [drm] number of scanouts: 1483gitlab # [ 1.597690] [drm] number of cap sets: 0484gitlab # [ 1.598555] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic485gitlab # [ 1.598566] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0486gitlab # [ 1.608578] Console: switching to colour frame buffer device 160x50487gitlab # [ 1.618201] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device488gitlab # [ 1.621037] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.489gitlab # [ 1.623740] systemd[1]: Starting Create Static Device Nodes in /dev...490gitlab # [ 1.648681] systemd[1]: Started Journal Service.491gitlab # [ 1.644707] systemd-modules-load[95]: Using 4 probe threads492gitlab # [ 1.648418] systemd-modules-load[95]: Module 'virtio_balloon' is built in493gitlab # [ 1.649753] systemd-modules-load[95]: Module 'virtio_console' is built in494gitlab # [ 1.651063] systemd-modules-load[95]: Module 'virtio_rng' is built in495gitlab # [ 1.652339] systemd-modules-load[95]: Inserted module 'dm_mod'496gitlab # [ 1.653477] systemd-modules-load[95]: Inserted module 'virtio_gpu'497gitlab # [ 1.655117] systemd[1]: Finished Load Kernel Modules.498gitlab # [ 1.656211] systemd[1]: Starting Apply Kernel Variables...499gitlab # [ 1.670042] systemd[1]: Finished Create Static Device Nodes in /dev.500gitlab # [ 1.672865] systemd[1]: Reached target Preparation for Local File Systems.501gitlab # [ 1.675499] systemd[1]: Reached target Local File Systems.502gitlab # [ 1.677685] systemd[1]: Starting Create System Files and Directories...503gitlab # [ 1.680293] systemd[1]: Starting Rule-based Manager for Device Events and Files...504gitlab # [ 1.684335] systemd[1]: Finished Apply Kernel Variables.505gitlab # [ 1.714956] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.738436] systemd-udevd[113]: Using default interface naming scheme 'v261'.507gitlab # [ 1.759768] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.792146] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.864878] systemd-vconsole-setup[131]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.869885] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.173734] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.176559] systemd[1]: Reached target System Initialization.513gitlab # [ 2.179038] systemd[1]: Reached target Basic System.514gitlab # [ 2.392593] (udev-worker)[146]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.398595] (udev-worker)[146]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.407083] systemd[1]: Found device /dev/disk/by-label/nixos.517gitlab # [ 2.408811] systemd[1]: Reached target Initrd Root Device.518gitlab # [ 2.410310] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...519gitlab # [ 2.427939] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line.520gitlab # [ 2.431727] systemd[1]: Found device /dev/disk/by-label/nix-store.521gitlab # [ 2.468853] systemd-fsck[153]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.476510] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.549306] systemd[1]: Mounting /sysroot...524gitlab # [ 2.588849] EXT4-fs (vda): mounted filesystem d8c68e5c-7e23-4bf2-a77d-e7ad2822ac6a r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.586810] systemd[1]: Mounted /sysroot.526gitlab # [ 2.589250] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.590387] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.592086] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.594639] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.616917] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.620971] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.628208] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.643042] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload.534gitlab # [ 2.659474] systemd[1]: Mounted /sysroot/nix/.rw-store.535gitlab # [ 2.661083] systemd[1]: Mounted /sysroot/run.536gitlab # [ 2.662297] systemd[1]: Mounted /sysroot/tmp/shared.537gitlab # [ 2.664209] systemd[1]: Mounted /sysroot/tmp/xchg.538gitlab # [ 2.685349] systemd[1]: Starting rw-sysroot-nix-store.service...539gitlab # [ 2.693277] erofs (device vdb): mounted with root inode @ nid 128.540gitlab # [ 2.690236] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)...541gitlab # [ 2.693030] systemd[1]: Reloading...542gitlab # [ 2.790462] systemd[1]: Reloading finished in 98 ms.543gitlab # [ 2.839686] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace...544gitlab # [ 2.842629] systemd[1]: Mounted /sysroot/nix/.ro-store.545gitlab # [ 2.843653] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.546gitlab # [ 2.847390] systemd[1]: Finished rw-sysroot-nix-store.service.547gitlab # [ 2.848343] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace...548gitlab # [ 2.849582] systemd[1]: initrd-parse-etc.service: Deactivated successfully.549gitlab # [ 2.850541] systemd[1]: Finished Mountpoints Configured in the Real Root.550gitlab # [ 2.851601] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.551gitlab # [ 2.853329] systemd[1]: Starting rw-sysroot-nix-store.service...552gitlab # [ 2.870748] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.553gitlab # [ 2.871947] systemd[1]: Finished rw-sysroot-nix-store.service.554gitlab # [ 2.977754] (udev-worker)[135]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory555gitlab # [ 2.982791] (udev-worker)[135]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory556gitlab # [ 2.987597] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.557gitlab # [ 2.988717] systemd[1]: Stopped Virtual Console Setup.558gitlab # [ 2.989456] systemd[1]: Stopping Virtual Console Setup...559gitlab # [ 2.990865] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 3.016867] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.561gitlab # [ 3.018079] systemd[1]: Stopped Virtual Console Setup.562gitlab # [ 3.018888] systemd[1]: Starting Virtual Console Setup...563gitlab # [ 3.042456] systemd-vconsole-setup[199]: Configuration of first virtual console was skipped, ignoring remaining ones.564gitlab # [ 3.044600] systemd[1]: Finished Virtual Console Setup.565gitlab # [ 3.553528] systemd[1]: Mounting /sysroot/nix/store...566gitlab # [ 3.634836] systemd[1]: Mounted /sysroot/nix/store.567gitlab # [ 3.637230] systemd[1]: Reached target Initrd File Systems.568gitlab # [ 3.639779] systemd[1]: Starting Find NixOS closure...569gitlab # [ 3.642627] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...570gitlab # [ 3.692218] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.571gitlab # [ 3.695972] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.572gitlab # [ 3.716724] systemd[1]: Finished Find NixOS closure.573gitlab # [ 3.719059] systemd[1]: Reached target Initrd Default Target.574gitlab # [ 3.721660] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...575gitlab # [ 3.768320] systemd[1]: Stopped target Initrd Default Target.576gitlab # [ 3.770881] systemd[1]: Stopped target Basic System.577gitlab # [ 3.773208] systemd[1]: Stopped target Initrd Root Device.578gitlab # [ 3.775571] systemd[1]: Stopped target Path Units.579gitlab # [ 3.777810] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.580gitlab # [ 3.781045] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.581gitlab # [ 3.784357] systemd[1]: Stopped target Slice Units.582gitlab # [ 3.786506] systemd[1]: Stopped target Socket Units.583gitlab # [ 3.789972] systemd[1]: Stopped target System Initialization.584gitlab # [ 3.792587] systemd[1]: Stopped target Swaps.585gitlab # [ 3.797679] systemd[1]: Stopped target Timer Units.586gitlab # [ 3.801174] systemd[1]: dbus.socket: Deactivated successfully.587gitlab # [ 3.805436] systemd[1]: Closed D-Bus System Message Bus Socket.588gitlab # [ 3.808069] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.589gitlab # [ 3.816406] systemd[1]: Stopped Find NixOS closure.590gitlab # [ 3.818153] systemd[1]: Starting Load Kernel Module 9pnet_virtio...591gitlab # [ 3.820268] systemd[1]: Starting rw-sysroot-nix-store.service...592gitlab # [ 3.822003] systemd[1]: systemd-sysctl.service: Deactivated successfully.593gitlab # [ 3.823862] systemd[1]: Stopped Apply Kernel Variables.594gitlab # [ 3.825408] systemd[1]: systemd-modules-load.service: Deactivated successfully.595gitlab # [ 3.827302] systemd[1]: Stopped Load Kernel Modules.596gitlab # [ 3.828797] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.597gitlab # [ 3.830775] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.598gitlab # [ 3.832919] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.599gitlab # [ 3.834813] systemd[1]: Stopped Create System Files and Directories.600gitlab # [ 3.836365] systemd[1]: Stopped target Local File Systems.601gitlab # [ 3.837690] systemd[1]: Stopped target Preparation for Local File Systems.602gitlab # [ 3.839312] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.603gitlab # [ 3.841099] systemd[1]: Stopped Coldplug All udev Devices.604gitlab # [ 3.842419] systemd[1]: Stopping Rule-based Manager for Device Events and Files...605gitlab # [ 3.844192] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.606gitlab # [ 3.845936] systemd[1]: Stopped Virtual Console Setup.607gitlab # [ 3.847197] systemd[1]: initrd-cleanup.service: Deactivated successfully.608gitlab # [ 3.848893] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.609gitlab # [ 3.850441] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.610gitlab # [ 3.852153] systemd[1]: Finished rw-sysroot-nix-store.service.611gitlab # [ 3.853543] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.612gitlab # [ 3.855265] systemd[1]: Finished Load Kernel Module 9pnet_virtio.613gitlab # [ 3.856788] systemd[1]: systemd-udevd.service: Deactivated successfully.614gitlab # [ 3.858352] systemd[1]: Stopped Rule-based Manager for Device Events and Files.615gitlab # [ 3.860065] systemd[1]: systemd-udevd.service: Consumed 2.234s CPU time over 2.154s wall clock time, 31.6M memory peak.616gitlab # [ 3.862478] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.617gitlab # [ 3.864269] systemd[1]: Closed udev Control Socket.618gitlab # [ 3.866808] systemd[1]: Starting Cleanup udev Database...619gitlab # [ 3.868186] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.620gitlab # [ 3.869988] systemd[1]: Stopped Create Static Device Nodes in /dev.621gitlab # [ 3.871476] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.622gitlab # [ 3.873517] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.623gitlab # [ 3.875218] systemd[1]: kmod-static-nodes.service: Deactivated successfully.624gitlab # [ 3.876941] systemd[1]: Stopped Create List of Static Device Nodes.625gitlab # [ 3.938879] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.626gitlab # [ 3.941131] systemd[1]: Finished Cleanup udev Database.627gitlab # [ 3.942650] systemd[1]: Reached target Switch Root.628gitlab # [ 3.944704] systemd[1]: Starting NixOS Activation...629gitlab # [ 4.131422] initrd-nixos-activation-start[225]: booting system configuration /nix/store/hsdm6hk3kxdrv96gkn9jddzas26sjqg4-nixos-system-gitlab-test630gitlab # [ 4.203143] initrd-nixos-activation-start[225]: running activation script...631gitlab # [ 4.524682] initrd-nixos-activation-start[248]: setting up /etc...632gitlab # [ 4.678886] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.633gitlab # [ 4.682361] systemd[1]: Finished NixOS Activation.634gitlab # [ 4.684602] systemd[1]: Starting Switch Root...635gitlab # [ 4.739324] systemd[1]: Switching root.636gitlab # [ 4.884287] systemd-journald[94]: Received SIGTERM from PID 1 (systemd).637gitlab # [ 5.505365] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)638gitlab # [ 5.505505] systemd[1]: Detected virtualization qemu.639gitlab # [ 5.505584] systemd[1]: Detected architecture arm64.640gitlab # [ 5.505759] systemd[1]: Detected first boot.641gitlab # [ 5.507139] systemd[1]: Initializing machine ID from random generator.642gitlab # [ 5.744887] systemd[1]: bpf-restrict-fs: LSM BPF program attached643gitlab # [ 5.867873] systemd[1]: Applying preset policy.644gitlab # [ 6.051865] systemd[1]: Populated /etc with preset unit settings.645gitlab # [ 6.206874] systemd[1]: initrd-switch-root.service: Deactivated successfully.646gitlab # [ 6.207139] systemd[1]: Stopped initrd-switch-root.service.647gitlab # [ 6.208949] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.648gitlab # [ 6.214021] systemd[1]: Created slice Slice /system/getty.649gitlab # [ 6.220506] systemd[1]: Created slice GitLab DevOps Platform Slice.650gitlab # [ 6.224185] systemd[1]: Created slice User and Session Slice.651gitlab # [ 6.224426] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.652gitlab # [ 6.225405] systemd[1]: Started Forward Password Requests to Wall Directory Watch.653gitlab # [ 6.226276] systemd[1]: Expecting device /dev/hvc0...654gitlab # [ 6.227154] systemd[1]: Expecting device /dev/ttyAMA0...655gitlab # [ 6.228079] systemd[1]: Reached target Local Encrypted Volumes.656gitlab # [ 6.232531] systemd[1]: Stopped target initrd-fs.target.657gitlab # [ 6.233761] systemd[1]: Stopped target initrd-root-fs.target.658gitlab # [ 6.234831] systemd[1]: Stopped target initrd-switch-root.target.659gitlab # [ 6.235975] systemd[1]: Reached target Virtual Machines and Containers.660gitlab # [ 6.237147] systemd[1]: Reached target Path Units.661gitlab # [ 6.238261] systemd[1]: Reached target Remote File Systems.662gitlab # [ 6.239372] systemd[1]: Reached target Slice Units.663gitlab # [ 6.240461] systemd[1]: Reached target Swaps.664gitlab # [ 6.244481] systemd[1]: Listening on Query the User Interactively for a Password.665gitlab # [ 6.248664] systemd[1]: Listening on Process Core Dump Socket.666gitlab # [ 6.251686] systemd[1]: Listening on Credential Encryption/Decryption.667gitlab # [ 6.255335] systemd[1]: Listening on Factory Reset Management.668gitlab # [ 6.256688] systemd[1]: Listening on Hostname Service Socket.669gitlab # [ 6.266788] systemd[1]: Starting Journal Log Access Socket...670gitlab # [ 6.268853] systemd[1]: Listening on Journal Audit Socket.671gitlab # [ 6.272896] systemd[1]: Listening on Console Output Muting Service Socket.672gitlab # [ 6.274527] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.673gitlab # [ 6.275692] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os674gitlab # [ 6.276717] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki675gitlab # [ 6.282366] systemd[1]: Listening on Disk Repartitioning Service Socket.676gitlab # [ 6.283619] systemd[1]: Listening on udev Control Socket.677gitlab # [ 6.284908] systemd[1]: Listening on udev Varlink Socket.678gitlab # [ 6.294372] systemd[1]: Mounting Huge Pages File System...679gitlab # [ 6.302388] systemd[1]: Mounting POSIX Message Queue File System...680gitlab # [ 6.313680] systemd[1]: Mounting Kernel Debug File System...681gitlab # [ 6.320556] systemd[1]: Mounting Kernel Trace File System...682gitlab # [ 6.332517] systemd[1]: Starting Create List of Static Device Nodes...683gitlab # [ 6.344610] systemd[1]: Starting Load Kernel Module 9pnet_virtio...684gitlab # [ 6.348072] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs685gitlab # [ 6.356888] systemd[1]: Mounting Kernel Configuration File System...686gitlab # [ 6.359885] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm687gitlab # [ 6.362103] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore688gitlab # [ 6.370207] systemd[1]: Starting Load Kernel Module fuse...689gitlab # [ 6.371016] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67690gitlab # [ 6.381565] systemd[1]: Starting Journal Service...691gitlab # [ 6.385402] systemd[1]: Starting Load Kernel Modules...692gitlab # [ 6.397182] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...693gitlab # [ 6.401899] systemd[1]: Starting Remount Root and Kernel File Systems...694gitlab # [ 6.403742] fuse: init (API version 7.45)695gitlab # [ 6.407461] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os696gitlab # [ 6.420442] systemd[1]: Starting Coldplug All udev Devices...697gitlab # [ 6.426208] systemd[1]: Listening on Journal Log Access Socket.698gitlab # [ 6.427032] systemd[1]: Mounted Huge Pages File System.699gitlab # [ 6.427893] systemd[1]: Mounted POSIX Message Queue File System.700gitlab # [ 6.430273] systemd[1]: Mounted Kernel Debug File System.701gitlab # [ 6.432088] systemd[1]: Mounted Kernel Trace File System.702gitlab # [ 6.433403] systemd[1]: Finished Create List of Static Device Nodes.703gitlab # [ 6.434285] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.704gitlab # [ 6.435686] systemd[1]: Finished Load Kernel Module 9pnet_virtio.705gitlab # [ 6.438661] systemd-journald[319]: Collecting audit messages is enabled.706gitlab # [ 6.439764] systemd[1]: Mounted Kernel Configuration File System.707gitlab # [ 6.440940] systemd[1]: modprobe@fuse.service: Deactivated successfully.708gitlab # [ 6.441864] systemd[1]: Finished Load Kernel Module fuse.709gitlab # [ 6.443005] systemd[1]: Finished Load Kernel Modules.710gitlab # [ 6.460514] systemd[1]: Mounting FUSE Control File System...711gitlab # [ 6.456896] systemd[1]: Queued start job for default target Multi-User System.712gitlab # [ 6.460670] EXT4-fs (vda): re-mounted d8c68e5c-7e23-4bf2-a77d-e7ad2822ac6a.713gitlab # [ 6.463649] systemd[1]: Unnecessary job was removed for /dev/vdb.714gitlab # [ 6.465109] systemd[1]: systemd-journald.service: Deactivated successfully.715gitlab # [ 6.466708] systemd-modules-load[320]: Using 3 probe threads716gitlab # [ 6.467955] systemd-modules-load[320]: Module 'atkbd' is built in717gitlab # [ 6.469378] systemd-modules-load[320]: Module 'loop' is built in718gitlab # [ 6.470643] systemd-modules-load[320]: Inserted module 'tls'719gitlab # [ 6.472600] systemd[1]: Starting Firewall...720gitlab # [ 6.494917] systemd[1]: Starting Apply Kernel Variables...721gitlab # [ 6.504408] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...722gitlab # [ 6.506617] systemd[1]: Started Journal Service.723gitlab # [ 6.505803] systemd-oomd[322]: No swap; memory pressure usage will be degraded724gitlab # [ 6.509638] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.725gitlab # [ 6.511703] systemd[1]: Finished Remount Root and Kernel File Systems.726gitlab # [ 6.514249] systemd[1]: Mounted FUSE Control File System.727gitlab # [ 6.529193] systemd[1]: Listening on Disk Image Download Service Socket.728gitlab # [ 6.532763] systemd[1]: Starting Flush Journal to Persistent Storage...729gitlab # [ 6.534029] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore730gitlab # [ 6.544856] systemd[1]: Starting Load/Save OS Random Seed...731gitlab # [ 6.545913] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os732gitlab # [ 6.548794] systemd[1]: Finished Apply Kernel Variables.733gitlab # [ 6.556107] systemd-journald[319]: Received client request to flush runtime journal.734gitlab # [ 6.579979] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.735gitlab # [ 6.583192] systemd[1]: Starting Create Static Device Nodes in /dev...736gitlab # [ 6.584398] systemd[1]: Finished Load/Save OS Random Seed.737gitlab # [ 6.585231] systemd[1]: Reached target First Boot Complete.738gitlab # [ 6.586027] systemd[1]: Finished Flush Journal to Persistent Storage.739gitlab # [ 6.586958] systemd[1]: Finished Create Static Device Nodes in /dev.740gitlab # [ 6.587873] systemd[1]: Reached target Preparation for Local File Systems.741gitlab # [ 6.589314] systemd[1]: Starting Rule-based Manager for Device Events and Files...742gitlab # [ 6.646701] systemd-udevd[372]: Using default interface naming scheme 'v261'.743gitlab # [ 6.686556] systemd[1]: Started Rule-based Manager for Device Events and Files.744gitlab # [ 6.906172] systemd[1]: Finished Firewall.745gitlab # [ 7.059245] systemd[1]: Finished Coldplug All udev Devices.746gitlab # [ 7.115781] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs747gitlab # [ 7.156795] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.748gitlab # [ 7.192070] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.749gitlab # [ 7.199723] (udev-worker)[471]: Network interface NamePolicy= disabled on kernel command line.750gitlab # [ 7.211442] (udev-worker)[409]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.751gitlab # [ 7.216833] (udev-worker)[409]: Network interface NamePolicy= disabled on kernel command line.752gitlab # [ 7.233481] systemd[1]: Mounting /run/wrappers...753gitlab # [ 7.246060] systemd[1]: Condition check resulted in Virtio network device being skipped.754gitlab # [ 7.247525] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore755gitlab # [ 7.250007] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67756gitlab # [ 7.252259] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore757gitlab # [ 7.253830] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os758gitlab # [ 7.255127] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os759gitlab # [ 7.266955] systemd[1]: Mounted /run/wrappers.760gitlab # [ 7.270252] systemd[1]: Reached target Local File Systems.761gitlab # [ 7.271100] systemd[1]: Listening on Boot Loader Control Service Socket.762gitlab # [ 7.272227] systemd[1]: Starting register-nix-paths.service...763gitlab # [ 7.274837] systemd[1]: Starting Create SUID/SGID Wrappers...764gitlab # [ 7.275829] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.765gitlab # [ 7.282467] systemd[1]: Starting Save Transient machine-id to Disk...766gitlab # [ 7.302997] systemd[1]: Starting Create System Files and Directories...767gitlab # [ 7.352962] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.768gitlab # [ 7.354756] systemd[1]: Finished Save Transient machine-id to Disk.769gitlab # [ 7.355776] systemd[1]: Finished Create System Files and Directories.770gitlab # [ 7.359756] systemd[1]: Starting Rebuild Journal Catalog...771gitlab # [ 7.422479] systemd[1]: Starting Record System Boot/Shutdown in UTMP...772gitlab # [ 7.450385] systemd[1]: Finished Rebuild Journal Catalog.773gitlab # [ 7.471368] systemd[1]: Starting Update is Completed...774gitlab # [ 7.502532] systemd[1]: Finished Record System Boot/Shutdown in UTMP.775gitlab # [ 7.526217] mousedev: PS/2 mouse device common for all mice776gitlab # [ 7.524200] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.777gitlab # [ 7.526705] systemd[1]: Finished Create SUID/SGID Wrappers.778gitlab # [ 7.528195] systemd[1]: suid-sgid-wrappers.service: Consumed 154ms CPU time over 247ms wall clock time, 6.3M memory peak, 1.9M read from disk.779gitlab # [ 7.532506] systemd[1]: Finished Update is Completed.780gitlab # [ 7.786983] systemd[1]: Finished register-nix-paths.service.781gitlab # [ 7.788662] systemd[1]: Reached target System Initialization.782gitlab # [ 7.789841] systemd[1]: Started Discard unused filesystem blocks once a week.783gitlab # [ 7.790840] systemd[1]: Started logrotate.timer.784gitlab # [ 7.791715] systemd[1]: Started Daily Cleanup of Temporary Directories.785gitlab # [ 7.792903] systemd[1]: Reached target Timer Units.786gitlab # [ 7.793716] systemd[1]: Listening on D-Bus System Message Bus Socket.787gitlab # [ 7.794601] systemd[1]: Listening on Nix Daemon Socket.788gitlab # [ 7.813176] systemd[1]: Starting nixbot CI service socket...789gitlab # [ 7.814491] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.790gitlab # [ 7.815761] systemd[1]: Listening on nixbot CI service socket.791gitlab # [ 7.817971] systemd[1]: Reached target Socket Units.792gitlab # [ 7.820183] systemd[1]: Starting D-Bus System Message Bus...793gitlab # [ 7.862202] dbus-broker-launch[561]: Looking up NSS user entry for 'systemd-timesync'...794gitlab # [ 7.863909] dbus-broker-launch[561]: NSS returned no entry for 'systemd-timesync'795gitlab # [ 7.865431] dbus-broker-launch[561]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"796gitlab # [ 7.868101] systemd[1]: Started D-Bus System Message Bus.797gitlab # [ 7.884586] dbus-broker-launch[561]: Ready798gitlab # [ 7.888249] systemd[1]: Reached target Basic System.799gitlab # [ 7.889260] systemd[1]: Started backdoor.service.800gitlab # [ 7.895089] systemd[1]: Starting gitlab-config.service...801gitlab # [ 7.912612] systemd[1]: Starting Import lastlog data into lastlog2 database...802gitlab # connecting to host...803gitlab: Guest shell says: b'Spawning backdoor root shell...\n'804gitlab # [ 7.964726] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3805gitlab: connected to guest root shell806gitlab: (connecting took 8.40 seconds)807gitlab: (finished: waiting for the VM to finish booting, in 8.40 seconds)808gitlab # [ 7.979595] systemd[1]: Starting Logrotate configuration check...809gitlab # [ 7.995921] systemd[1]: Starting Name Service Cache Daemon (nsncd)...810gitlab # [ 8.001590] systemd[1]: Starting Post-Boot Actions...811gitlab # [ 8.007153] logrotate[577]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.812gitlab # [ 8.009546] logrotate[577]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf813gitlab # [ 8.010793] logrotate[577]: note: 'monthly' overrides previously specified 'weekly'814gitlab # [ 8.011965] logrotate[577]: Reading state from file: /var/lib/logrotate.status815gitlab # [ 8.013631] logrotate[577]: state file /var/lib/logrotate.status does not exist816gitlab # [ 8.014614] logrotate[577]: Allocating hash table for state file, size 64 entries817gitlab # [ 8.017756] logrotate[577]: Handling 4 logs818gitlab # [ 8.018486] logrotate[577]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed819gitlab # [ 8.020787] logrotate[577]: considering log /var/log/btmp820gitlab # [ 8.021573] logrotate[577]: Creating new state821gitlab # [ 8.022195] logrotate[577]: Now: 2026-08-28 11:35822gitlab # [ 8.022883] logrotate[577]: Last rotated at 2026-08-28 11:00823gitlab # [ 8.023690] logrotate[577]: log does not need rotating (log has already been rotated)824gitlab # [ 8.028498] logrotate[577]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed825gitlab # [ 8.030713] logrotate[577]: considering log /var/log/wtmp826gitlab # [ 8.031499] logrotate[577]: Creating new state827gitlab # [ 8.033317] logrotate[577]: Now: 2026-08-28 11:35828gitlab # [ 8.034125] logrotate[577]: Last rotated at 2026-08-28 11:00829gitlab # [ 8.035584] logrotate[577]: log does not need rotating (log has already been rotated)830gitlab # [ 8.040393] logrotate[577]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed831gitlab # [ 8.042298] logrotate[577]: switching euid from 0 to 165 and egid from 0 to 165 (pid 577)832gitlab # [ 8.043455] logrotate[577]: considering log /var/gitlab/state/log/*.log833gitlab # [ 8.044402] logrotate[577]: log /var/gitlab/state/log/*.log does not exist -- skipping834gitlab # [ 8.045549] logrotate[577]: Creating new state835gitlab # [ 8.046168] logrotate[577]: switching euid from 165 to 0 and egid from 165 to 0 (pid 577)836gitlab # [ 8.048165] logrotate[577]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed837gitlab # [ 8.049852] logrotate[577]: switching euid from 0 to 60 and egid from 0 to 60 (pid 577)838gitlab # [ 8.050945] logrotate[577]: considering log /var/log/nginx/*.log839gitlab # [ 8.051783] logrotate[577]: log /var/log/nginx/*.log does not exist -- skipping840gitlab # [ 8.056987] logrotate[577]: Creating new state841gitlab # [ 8.057713] logrotate[577]: not running postrotate script, since no logs were rotated842gitlab # [ 8.058733] logrotate[577]: switching euid from 60 to 0 and egid from 60 to 0 (pid 577)843gitlab # [ 8.060486] nsncd[578]: Aug 28 11:35:10.941 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"844gitlab # [ 8.062794] systemd[1]: Started Reset console on configuration changes.845gitlab # [ 8.064115] systemd[1]: Starting resolvconf update...846gitlab # [ 8.065025] systemd[1]: Finished Logrotate configuration check.847gitlab # [ 8.066089] systemd[1]: Finished Post-Boot Actions.848gitlab # [ 8.066938] systemd[1]: Started Name Service Cache Daemon (nsncd).849gitlab # [ 8.067907] systemd[1]: Finished Import lastlog data into lastlog2 database.850gitlab # [ 8.085910] systemd[1]: Reached target Host and Network Name Lookups.851gitlab # [ 8.087816] systemd[1]: Reached target User and Group Name Lookups.852gitlab # [ 8.092936] systemd[1]: Starting User Login Management...853gitlab # [ 8.100000] systemd[1]: Starting Virtual Console Setup...854gitlab # [ 8.121645] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.855gitlab # [ 8.122816] systemd[1]: Stopped Virtual Console Setup.856gitlab # [ 8.125069] systemd[1]: Starting Virtual Console Setup...857gitlab # [ 8.129739] systemd[1]: Stopped target Host and Network Name Lookups.858gitlab # [ 8.131708] systemd[1]: Stopping Host and Network Name Lookups...859gitlab # [ 8.132971] systemd[1]: Stopped target User and Group Name Lookups.860gitlab # [ 8.133939] systemd[1]: Stopping User and Group Name Lookups...861gitlab # [ 8.134766] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...862gitlab # [ 8.135648] systemd[1]: nscd.service: Deactivated successfully.863gitlab # [ 8.137181] systemd[1]: Stopped Name Service Cache Daemon (nsncd).864gitlab # [ 8.138041] systemd[1]: nscd.service: Consumed 29ms CPU time over 139ms wall clock time, 7.4M memory peak, 5.8M read from disk.865gitlab # [ 8.144601] systemd[1]: Starting Name Service Cache Daemon (nsncd)...866gitlab # [ 8.172465] systemd-logind[633]: New seat seat0.867gitlab # [ 8.177410] systemd-logind[633]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)868gitlab # [ 8.182402] systemd-logind[633]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)869gitlab # [ 8.187571] systemd-logind[633]: Watching system buttons on /dev/input/event0 (gpio-keys)870gitlab # [ 8.188997] systemd[1]: Started User Login Management.871gitlab # [ 8.191487] systemd[1]: Starting linger-users.service...872gitlab # [ 8.194389] systemd[1]: Finished resolvconf update.873gitlab # [ 8.195106] systemd[1]: Reached target Preparation for Network.874gitlab # [ 8.195910] systemd[1]: Starting DHCP Client...875gitlab # [ 8.201293] systemd[1]: Starting Address configuration of eth1...876gitlab # [ 8.204426] nsncd[666]: Aug 28 11:35:11.096 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"877gitlab # [ 8.206634] systemd-vconsole-setup[653]: Configuration of first virtual console was skipped, ignoring remaining ones.878gitlab # [ 8.211939] systemd[1]: Starting Extra networking commands....879gitlab # [ 8.220769] systemd[1]: Started Name Service Cache Daemon (nsncd).880gitlab # [ 8.223604] systemd[1]: Finished Virtual Console Setup.881gitlab # [ 8.227543] systemd[1]: linger-users.service: Deactivated successfully.882gitlab # [ 8.229491] systemd[1]: Finished linger-users.service.883gitlab # [ 8.233250] systemd[1]: Reached target Host and Network Name Lookups.884gitlab # [ 8.234884] systemd[1]: Reached target User and Group Name Lookups.885gitlab # [ 8.244775] network-addresses-eth1-start[680]: adding address 192.168.1.1/24... done886gitlab # [ 8.249255] network-addresses-eth1-start[680]: adding address 2001:db8:1::1/64... done887gitlab # [ 8.264677] systemd[1]: Finished Address configuration of eth1.888gitlab # [ 8.303543] dhcpcd[701]: dhcpcd-10.3.2 starting889gitlab # [ 8.304591] systemd[1]: Finished Extra networking commands..890gitlab # [ 8.305515] systemd[1]: Reached target Network.891gitlab # [ 8.306537] dhcpcd[758]: dev: loaded udev892gitlab # [ 8.309213] systemd[1]: Starting gitlab-workhorse.service...893gitlab # [ 8.313596] systemd[1]: Starting Nginx Web Server...894gitlab # [ 8.324049] 8021q: 802.1Q VLAN Support v1.8895gitlab # [ 8.320592] systemd[1]: Starting PostgreSQL Server...896gitlab # [ 8.326540] 8021q: adding VLAN 0 to HW filter on device eth1897gitlab # [ 8.325978] systemd[1]: Starting Redis Server - redis-gitlab...898gitlab # [ 8.332334] systemd[1]: Starting Permit User Sessions...899gitlab # [ 8.343495] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.900gitlab # [ 8.359317] systemd[1]: Finished Permit User Sessions.901gitlab # [ 8.371907] cfg80211: Loading compiled-in X.509 certificates for regulatory database902gitlab # [ 8.373921] systemd[1]: Started Getty on tty1.903gitlab # [ 8.376919] systemd[1]: Reached target Login Prompts.[ 8.381640] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'904gitlab # [ 8.382156] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'905gitlab # 906gitlab # [ 8.383655] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2907gitlab # [ 8.383960] cfg80211: failed to load regulatory.db908gitlab # [ 8.429242] 8021q: adding VLAN 0 to HW filter on device eth0909gitlab # [ 8.426808] dhcpcd[758]: eth0: waiting for carrier910gitlab # [ 8.427930] dhcpcd[758]: eth0: carrier acquired911gitlab # [ 8.431465] dhcpcd[758]: DUID 00:01:00:01:32:24:2e:ef:52:54:00:12:34:56912gitlab # [ 8.432811] dhcpcd[758]: eth0: IAID 00:12:34:56913gitlab # [ 8.433549] dhcpcd[758]: eth0: adding address fe80::5054:ff:fe12:3456914gitlab # [ 8.493338] nginx-pre-start[795]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok915gitlab # [ 8.495173] nginx-pre-start[795]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful916gitlab # [ 8.502723] systemd[1]: Started Nginx Web Server.917gitlab # [ 8.528833] redis-server[790]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.918gitlab # [ 8.531032] redis-server[790]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo919gitlab # [ 8.533299] redis-server[790]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=790, just started920gitlab # [ 8.536360] postgresql-pre-start[797]: The files belonging to this database system will be owned by user "postgres".921gitlab # [ 8.539556] postgresql-pre-start[797]: This user must also own the server process.922gitlab # [ 8.541171] systemd[1]: Started Redis Server - redis-gitlab.923gitlab # [ 8.542116] redis-server[790]: Configuration loaded924gitlab # [ 8.543024] postgresql-pre-start[797]: The database cluster will be initialized with locale "en_US.UTF-8".925gitlab # [ 8.544630] postgresql-pre-start[797]: The default database encoding has accordingly been set to "UTF8".926gitlab # [ 8.546022] postgresql-pre-start[797]: The default text search configuration will be set to "english".927gitlab # [ 8.547316] postgresql-pre-start[797]: Data page checksums are enabled.928gitlab # [ 8.548834] postgresql-pre-start[797]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok929gitlab # [ 8.550366] postgresql-pre-start[797]: creating subdirectories ... ok930gitlab # [ 8.551278] postgresql-pre-start[797]: selecting dynamic shared memory implementation ... posix931gitlab # [ 8.552521] redis[790]: monotonic clock: ARM CNTVCT @ 1000 ticks/us932gitlab # [ 8.553645] redis[790]: Running mode=standalone, port=0.933gitlab # [ 8.554649] redis[790]: Server initialized934gitlab # [ 8.555487] redis[790]: Ready to accept connections unix935gitlab # [ 8.556526] redis[790]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.936gitlab # [ 8.585758] postgresql-pre-start[797]: selecting default "max_connections" ... 100937gitlab # [ 8.639074] postgresql-pre-start[797]: selecting default "shared_buffers" ... 128MB938gitlab # [ 8.779507] dhcpcd[758]: eth0: soliciting a DHCP lease939gitlab # [ 8.792921] dhcpcd[758]: eth0: offered 10.0.2.15 from 10.0.2.2940gitlab # [ 8.808475] dhcpcd[758]: eth0: probing address 10.0.2.15/24941gitlab # [ 8.949126] systemd[1]: Started gitlab-workhorse.service.942gitlab # [ 9.101725] postgresql-pre-start[797]: selecting default time zone ... UTC943gitlab # [ 9.104335] postgresql-pre-start[797]: creating configuration files ... ok944gitlab # [ 9.214898] gitlab-workhorse[820]: time="2026-08-28T11:35:12Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4945gitlab # [ 9.217878] gitlab-workhorse[820]: time="2026-08-28T11:35:12Z" level=info msg="Using redis/go-redis"946gitlab # [ 9.219967] gitlab-workhorse[820]: time="2026-08-28T11:35:12Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix947gitlab # [ 9.369032] postgresql-pre-start[797]: running bootstrap script ... ok948gitlab # [ 9.912500] dhcpcd[758]: eth0: soliciting an IPv6 router949gitlab # [ 9.913431] dhcpcd[758]: eth0: Router Advertisement from fe80::2950gitlab # [ 9.914306] dhcpcd[758]: eth0: adding address fec0::5054:ff:fe12:3456/64951gitlab # [ 9.915233] dhcpcd[758]: eth0: adding route to fec0::/64952gitlab # [ 9.915982] dhcpcd[758]: eth0: adding default route via fe80::2953gitlab # [ 10.071055] postgresql-pre-start[797]: performing post-bootstrap initialization ... ok954gitlab # [ 10.352705] postgresql-pre-start[797]: syncing data to disk ... ok955gitlab # [ 10.355507] postgresql-pre-start[797]: initdb: warning: enabling "trust" authentication for local connections956gitlab # [ 10.359895] postgresql-pre-start[797]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.957gitlab # [ 10.366806] postgresql-pre-start[797]: Success. You can now start the database server using:958gitlab # [ 10.370735] postgresql-pre-start[797]: pg_ctl -D /var/lib/postgresql/18 -l logfile start959gitlab # [ 10.549184] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit960gitlab # [ 10.554935] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432961gitlab # [ 10.558344] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432962gitlab # [ 10.561821] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"963gitlab # [ 10.570317] postgres[863]: [863] LOG: database system was shut down at 2026-08-28 11:35:12 GMT964gitlab # [ 10.581817] postgres[854]: [854] LOG: database system is ready to accept connections965gitlab # [ 10.587162] systemd[1]: Started PostgreSQL Server.966gitlab # [ 10.600233] systemd[1]: Starting PostgreSQL Setup Scripts...967gitlab # [ 10.967003] postgresql-setup-start[878]: CREATE DATABASE968gitlab # [ 11.012269] postgresql-setup-start[884]: CREATE ROLE969gitlab # [ 11.014075] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[876]: mkdir -p /var/gitlab/state/home/.ssh: OK970gitlab # [ 11.018419] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[876]: chmod 700 /var/gitlab/state/home/.ssh: OK971gitlab # [ 11.038036] postgresql-setup-start[888]: ALTER DATABASE972gitlab # [ 11.086864] postgresql-setup-start[898]: CREATE ROLE973gitlab # [ 11.088953] systemd[1]: Finished PostgreSQL Setup Scripts.974gitlab # [ 11.089817] systemd[1]: Reached target PostgreSQL.975gitlab # [ 11.091630] systemd[1]: Starting gitlab-postgresql.service...976gitlab # [ 11.120652] systemd[1]: Finished gitlab-config.service.977gitlab # [ 11.123896] systemd[1]: Started gitaly.service.978gitlab # [ 11.126703] systemd[1]: Starting gitlab-db-config.service...979gitlab # [ 11.269048] gitlab-postgresql-start[915]: CREATE DATABASE980gitlab # [ 11.329060] gitlab-postgresql-start[928]: CREATE EXTENSION981gitlab # [ 11.429683] gitlab-postgresql-start[935]: CREATE EXTENSION982gitlab # [ 11.431797] systemd[1]: Finished gitlab-postgresql.service.983gitlab # [ 11.748943] gitaly[907]: time="2026-08-28T11:35:14.642Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=907984gitlab # [ 11.753245] gitaly[907]: time="2026-08-28T11:35:14.642Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=907985gitlab # [ 11.755088] gitaly[907]: time="2026-08-28T11:35:14.642Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=907986gitlab # [ 11.760124] gitaly[907]: time="2026-08-28T11:35:14.642Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=907987gitlab # [ 11.762186] gitaly[907]: time="2026-08-28T11:35:14.642Z" level=info msg="Starting Gitaly" pid=907 version=988gitlab # [ 12.019603] gitaly[907]: time="2026-08-28T11:35:14.913Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=269 pid=907989gitlab # [ 12.024113] gitaly[907]: time="2026-08-28T11:35:14.913Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=907990gitlab # [ 12.027974] gitaly[907]: time="2026-08-28T11:35:14.913Z" level=info msg="finished initializing command factory" duration_ms=0 pid=907991gitlab # [ 12.032055] gitaly[907]: time="2026-08-28T11:35:14.913Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-907/git-exec-1267528718.d/git pid=907992gitlab # [ 12.037345] gitaly[907]: time="2026-08-28T11:35:14.919Z" level=info msg="finished detecting git version" duration_ms=5 pid=907993gitlab # [ 12.042786] gitaly[907]: time="2026-08-28T11:35:14.919Z" level=info msg="using Git version" pid=907 version=2.53.g994gitlab # [ 12.047618] gitaly[907]: time="2026-08-28T11:35:14.919Z" level=info msg="clearing disk cache object folder" pid=907 storage=default995gitlab # [ 12.051862] gitaly[907]: time="2026-08-28T11:35:14.919Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache396811967 pid=907 storage=default996gitlab # [ 12.056853] gitaly[907]: time="2026-08-28T11:35:14.919Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache396811967 pid=907 storage=default997gitlab # [ 12.063739] gitaly[907]: time="2026-08-28T11:35:14.919Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=907998gitlab # [ 12.070909] gitaly[907]: time="2026-08-28T11:35:14.919Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=907999gitlab # [ 12.079062] gitaly[907]: time="2026-08-28T11:35:14.920Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache396811967 clear_duration_ms=0 pid=907 storage=default1000gitlab # [ 12.086529] gitaly[907]: time="2026-08-28T11:35:14.935Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9071001gitlab # [ 12.091172] gitaly[907]: time="2026-08-28T11:35:14.935Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=907 schema=unix1002gitlab # [ 12.096321] gitaly[907]: time="2026-08-28T11:35:14.935Z" level=info msg="counting repositories" pid=9071003gitlab # [ 12.099559] gitaly[907]: time="2026-08-28T11:35:14.935Z" level=info msg="starting to count repositories" pid=907 storage_path=/var/gitlab/state/repositories1004gitlab # [ 12.104547] gitaly[907]: time="2026-08-28T11:35:14.935Z" level=info msg="listening at address" address=/run/gitaly/gitaly-907/sock.d/intern pid=907 schema=unix1005gitlab # [ 12.109443] gitaly[907]: time="2026-08-28T11:35:14.935Z" level=info msg="finished tempdir cleaner walk" pid=907 storage=default time_ms=01006gitlab # [ 12.113812] gitaly[907]: time="2026-08-28T11:35:14.935Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=907 storage_path=/var/gitlab/state/repositories1007gitlab # [ 12.119604] gitaly[907]: time="2026-08-28T11:35:14.936Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9071008gitlab # [ 12.124186] gitaly[907]: time="2026-08-28T11:35:14.941Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9071009gitlab # [ 12.128491] gitaly[907]: time="2026-08-28T11:35:14.941Z" level=info msg="Started Gitaly" duration_ms=298 pid=9071010gitlab # [ 12.132095] gitaly[907]: time="2026-08-28T11:35:14.941Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9071011gitlab # [ 12.136263] gitaly[907]: time="2026-08-28T11:35:14.941Z" level=info msg="maintenance: daily scheduled" pid=907 scheduled="2026-08-28 12:00:00 +0000 UTC"1012gitlab # [ 12.424820] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[913]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1013gitlab # [ 12.435869] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[913]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1014gitlab # [ 12.445924] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[913]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1015gitlab # [ 12.959409] dhcpcd[758]: eth0: leased 10.0.2.15 for 86400 seconds1016gitlab # [ 12.962378] dhcpcd[758]: eth0: adding route to 10.0.2.0/241017gitlab # [ 12.964764] dhcpcd[758]: eth0: adding default route via 10.0.2.21018gitlab # [ 13.125051] systemd[1]: Started DHCP Client.1019gitlab # [ 13.127433] systemd[1]: Reached target Network is Online.1020gitlab # [ 15.380111] gitaly[907]: time="2026-08-28T11:35:18.272Z" level=info msg="License database preloaded" duration_ms=3621 pid=9071021gitlab # [ 58.845244] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[913]: W, [2026-08-28T11:36:01.737402 #913] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1022gitlab # [ 59.280792] gitlab-workhorse[820]: time="2026-08-28T11:36:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1023gitlab # [ 61.535000] postgres[1437]: [1437] ERROR: relation "feature_gates" does not exist at character 5231024gitlab # [ 61.535481] postgres[1437]: [1437] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1025gitlab # [ 61.535608] postgres[1437]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1026gitlab # [ 61.535710] postgres[1437]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1027gitlab # [ 61.535828] postgres[1437]: attidentity AS identity,1028gitlab # [ 61.535951] postgres[1437]: attgenerated as attgenerated1029gitlab # [ 61.543906] postgres[1437]: FROM pg_attribute a1030gitlab # [ 61.554586] postgres[1437]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1031gitlab # [ 61.556429] postgres[1437]: LEFT JOIN pg_type t ON a.atttypid = t.oid1032gitlab # [ 61.557325] postgres[1437]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1033gitlab # [ 61.558694] postgres[1437]: WHERE a.attrelid = '"feature_gates"'::regclass1034gitlab # [ 61.559617] postgres[1437]: AND a.attnum > 0 AND NOT a.attisdropped1035gitlab # [ 61.560558] postgres[1437]: ORDER BY a.attnum1036gitlab # [ 61.561188] postgres[1437]:1037gitlab # [ 67.655789] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1497]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1038gitlab # [ 67.656311] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1497]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1039gitlab # [ 69.291063] gitlab-workhorse[820]: time="2026-08-28T11:36:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1040gitlab # [ 79.301389] gitlab-workhorse[820]: time="2026-08-28T11:36:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1041gitlab # [ 89.312477] gitlab-workhorse[820]: time="2026-08-28T11:36:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1042gitlab # [ 94.106841] postgres[1600]: [1600] ERROR: relation "table_without_models" does not exist1043gitlab # [ 94.107343] postgres[1600]: [1600] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1044gitlab # [ 99.322319] gitlab-workhorse[820]: time="2026-08-28T11:36:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1045gitlab # [ 109.330366] gitlab-workhorse[820]: time="2026-08-28T11:36:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1046gitlab # [ 119.338444] gitlab-workhorse[820]: time="2026-08-28T11:37:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1047gitlab # [ 125.959985] systemd[1]: Finished gitlab-db-config.service.1048gitlab # [ 125.964996] systemd[1]: Started gitlab-sidekiq.service.1049gitlab # [ 125.971329] systemd[1]: Starting gitlab.service...1050gitlab # [ 126.589656] bundle[1993]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1051gitlab # [ 126.590304] bundle[1993]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1052gitlab # [ 126.591865] bundle[1993]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1053gitlab # [ 129.345291] gitlab-workhorse[820]: time="2026-08-28T11:37:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1054gitlab # [ 131.757091] sidekiq-cluster[1992]: {"severity":"INFO","time":"2026-08-28T11:37:14.604Z","message":"Starting cluster with 1 processes"}1055gitlab # [ 131.828631] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.672Z","pid":1993,"message":"* Enabling systemd notification integration"}1056gitlab # [ 131.830437] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.721Z","pid":1993,"message":"Puma starting in cluster mode..."}1057gitlab # [ 131.831897] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.721Z","pid":1993,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1058gitlab # [ 131.836390] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.721Z","pid":1993,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1059gitlab # [ 131.838493] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.721Z","pid":1993,"message":"* Min threads: 0"}1060gitlab # [ 131.839822] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.721Z","pid":1993,"message":"* Max threads: 4"}1061gitlab # [ 131.841339] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.721Z","pid":1993,"message":"* Environment: production"}1062gitlab # [ 131.842856] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.727Z","pid":1993,"message":"* Master PID: 1993"}1063gitlab # [ 131.844251] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.727Z","pid":1993,"message":"* Workers: 2"}1064gitlab # [ 131.845583] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.727Z","pid":1993,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1065gitlab # [ 131.847280] bundle[1993]: {"timestamp":"2026-08-28T11:37:14.727Z","pid":1993,"message":"* Preloading application"}1066gitlab # [ 139.344495] gitlab-workhorse[820]: time="2026-08-28T11:37:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1067gitlab # [ 149.354458] gitlab-workhorse[820]: time="2026-08-28T11:37:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1068gitlab # [ 159.362197] gitlab-workhorse[820]: time="2026-08-28T11:37:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1069gitlab # [ 169.372399] gitlab-workhorse[820]: time="2026-08-28T11:37:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1070gitlab # [ 174.710652] bundle[1993]: W, [2026-08-28T11:37:57.603175 #1993] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1071gitlab # [ 178.945022] sidekiq-cluster[2043]: 2026-08-28T11:38:01.837Z pid=2043 tid=wnn INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2043, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426160128}1072gitlab # [ 179.252203] sidekiq-cluster[2043]: W, [2026-08-28T11:38:02.145829 #2043] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1073gitlab # [ 179.382288] gitlab-workhorse[820]: time="2026-08-28T11:38:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1074gitlab # [ 179.407281] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:02.300Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1075gitlab # [ 182.965618] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.858Z","message":"GitLab reliable fetch activated!"}1076gitlab # [ 182.970740] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.858Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1077gitlab # [ 183.000545] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.894Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1078gitlab # [ 183.007505] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.897Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1079gitlab # [ 183.014763] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.900Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1080gitlab # [ 183.021465] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.903Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1081gitlab # [ 183.028916] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.906Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1082gitlab # [ 183.035476] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.909Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1083gitlab # [ 183.041523] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.911Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1084gitlab # [ 183.046283] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.913Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1085gitlab # [ 183.050820] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.915Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1086gitlab # [ 183.055148] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.917Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1087gitlab # [ 183.059691] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.919Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1088gitlab # [ 183.063756] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.922Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1089gitlab # [ 183.067276] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.924Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1090gitlab # [ 183.070851] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.927Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1091gitlab # [ 183.074261] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.929Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1092gitlab # [ 183.077536] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.931Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1093gitlab # [ 183.080445] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.934Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1094gitlab # [ 183.083155] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.937Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1095gitlab # [ 183.086288] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.941Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1096gitlab # [ 183.089109] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.944Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1097gitlab # [ 183.092035] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.947Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1098gitlab # [ 183.094980] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.950Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1099gitlab # [ 183.097665] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.953Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1100gitlab # [ 183.100206] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.955Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1101gitlab # [ 183.102718] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.958Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1102gitlab # [ 183.105160] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.961Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1103gitlab # [ 183.107380] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.964Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1104gitlab # [ 183.109802] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.967Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1105gitlab # [ 183.112114] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.971Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1106gitlab # [ 183.114471] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.974Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1107gitlab # [ 183.116672] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.977Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1108gitlab # [ 183.119037] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.979Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1109gitlab # [ 183.121238] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.982Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1110gitlab # [ 183.123421] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.985Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1111gitlab # [ 183.125692] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.988Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1112gitlab # [ 183.127785] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.990Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1113gitlab # [ 183.130117] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.992Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1114gitlab # [ 183.132320] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.995Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1115gitlab # [ 183.134390] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:05.997Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1116gitlab # [ 183.136465] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.000Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1117gitlab # [ 183.138632] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.002Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1118gitlab # [ 183.140818] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.004Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1119gitlab # [ 183.143034] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.006Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1120gitlab # [ 183.145209] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.009Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1121gitlab # [ 183.147403] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.012Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1122gitlab # [ 183.149808] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.014Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1123gitlab # [ 183.151942] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.017Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1124gitlab # [ 183.154260] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.019Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1125gitlab # [ 183.156391] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.022Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1126gitlab # [ 183.158479] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.025Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1127gitlab # [ 183.160637] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.029Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1128gitlab # [ 183.162708] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.032Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1129gitlab # [ 183.164882] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.047Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1130gitlab # [ 183.167228] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.050Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1131gitlab # [ 183.169550] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.053Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1132gitlab # [ 183.171695] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.056Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1133gitlab # [ 183.174056] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.059Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1134gitlab # [ 183.176372] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.062Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1135gitlab # [ 183.178612] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.064Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1136gitlab # [ 183.180942] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.066Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1137gitlab # [ 183.182986] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.068Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1138gitlab # [ 183.186132] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.070Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1139gitlab # [ 183.263855] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.157Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1140gitlab # [ 183.271754] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.165Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1141gitlab # [ 183.278992] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.169Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1142gitlab # [ 183.285847] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.172Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1143gitlab # [ 183.292292] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.175Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1144gitlab # [ 183.298970] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.177Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1145gitlab # [ 183.306087] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.180Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1146gitlab # [ 183.312331] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.182Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1147gitlab # [ 183.317678] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.185Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1148gitlab # [ 183.322540] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.188Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1149gitlab # [ 183.327111] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.191Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1150gitlab # [ 183.331679] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.193Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1151gitlab # [ 183.335986] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.195Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1152gitlab # [ 183.339806] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.198Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1153gitlab # [ 183.343535] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.200Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1154gitlab # [ 183.347450] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.203Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1155gitlab # [ 183.350780] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.206Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1156gitlab # [ 183.353989] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.208Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1157gitlab # [ 183.357191] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.210Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1158gitlab # [ 183.360310] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.213Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1159gitlab # [ 183.363193] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.216Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1160gitlab # [ 183.366050] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.219Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1161gitlab # [ 183.368890] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.222Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1162gitlab # [ 183.371421] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.225Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1163gitlab # [ 183.374232] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.227Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1164gitlab # [ 183.376798] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.230Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1165gitlab # [ 183.379273] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.232Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1166gitlab # [ 183.381855] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.234Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1167gitlab # [ 183.384350] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.237Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1168gitlab # [ 183.386650] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.239Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1169gitlab # [ 183.389048] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.243Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1170gitlab # [ 183.392561] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.245Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1171gitlab # [ 183.394773] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.249Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1172gitlab # [ 183.397821] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.252Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1173gitlab # [ 183.400668] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.256Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1174gitlab # [ 183.403421] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.258Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1175gitlab # [ 183.406331] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.270Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1176gitlab # [ 183.409131] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.272Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1177gitlab # [ 183.411807] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.274Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1178gitlab # [ 183.414810] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.279Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1179gitlab # [ 183.417556] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.282Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1180gitlab # [ 183.420239] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.285Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1181gitlab # [ 183.423137] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.288Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1182gitlab # [ 183.426207] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.290Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1183gitlab # [ 183.428862] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.293Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1184gitlab # [ 183.431640] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.297Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1185gitlab # [ 183.434525] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:06.299Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1186gitlab # [ 189.390937] gitlab-workhorse[820]: time="2026-08-28T11:38:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1187gitlab # [ 199.398480] gitlab-workhorse[820]: time="2026-08-28T11:38:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1188gitlab # [ 209.406380] gitlab-workhorse[820]: time="2026-08-28T11:38:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1189gitlab # [ 219.417316] gitlab-workhorse[820]: time="2026-08-28T11:38:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1190gitlab # [ 224.630431] postgres[2825]: [2825] ERROR: relation "table_without_models" does not exist1191gitlab # [ 224.630871] postgres[2825]: [2825] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1192gitlab # [ 229.428207] gitlab-workhorse[820]: time="2026-08-28T11:38:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1193gitlab # [ 229.658521] postgres[2854]: [2854] ERROR: relation "table_without_models" does not exist1194gitlab # [ 229.662154] postgres[2854]: [2854] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1195gitlab # [ 230.784636] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:53.675Z","message":"Booted Rails 7.2.3.1 application in production environment"}1196gitlab # [ 230.786721] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:53.676Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1197gitlab # [ 230.792589] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:53.676Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1198gitlab # [ 230.794430] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:53.676Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1199gitlab # [ 230.796541] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:53.680Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1200gitlab # [ 233.637176] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:56.529Z","message":"Cleaning working queues"}1201gitlab # [ 233.640567] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:56.533Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"f4b05ca6ef1867a168f6aed4","created_at":"2026-08-28T11:37:08.243Z","trace_propagation_headers":{"sentry-trace":"9f10769a35e24222a2dd08c312b25e47-768b68734ad94182","baggage":"sentry-trace_id=9f10769a35e24222a2dd08c312b25e47,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"43efdf0bd8110727192924cdc60315a0","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-28T11:37:08.271Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-f4b05ca6ef1867a168f6aed4: start","job_status":"start","queue_duration_s":108.261705,"scheduling_latency_s":108.261787}1202gitlab # [ 234.033050] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:38:56.925Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"f4b05ca6ef1867a168f6aed4","created_at":"2026-08-28T11:37:08.243Z","trace_propagation_headers":{"sentry-trace":"9f10769a35e24222a2dd08c312b25e47-768b68734ad94182","baggage":"sentry-trace_id=9f10769a35e24222a2dd08c312b25e47,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"43efdf0bd8110727192924cdc60315a0","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-28T11:37:08.271Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-f4b05ca6ef1867a168f6aed4: done: 0.392253 sec","job_status":"done","queue_duration_s":108.261705,"scheduling_latency_s":108.261787,"redis_calls":29,"redis_duration_s":0.012704,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007653,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003173,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001878,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.206609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19626196,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.392253,"completed_at":"2026-08-28T11:38:56.925Z","load_balancing_strategy":"primary","db_duration_s":0.096782,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1203gitlab # [ 236.844849] bundle[1993]: {"timestamp":"2026-08-28T11:38:59.737Z","pid":1993,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1204gitlab # [ 236.845237] bundle[1993]: {"timestamp":"2026-08-28T11:38:59.737Z","pid":1993,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1205gitlab # [ 236.845459] bundle[1993]: {"timestamp":"2026-08-28T11:38:59.737Z","pid":1993,"message":"! #\u003cThread:0x0000f595de89a0c0 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1206gitlab # [ 236.845629] bundle[1993]: {"timestamp":"2026-08-28T11:38:59.737Z","pid":1993,"message":"! #\u003cThread:0x0000f595de1616f0 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1207gitlab # [ 236.845752] bundle[1993]: {"timestamp":"2026-08-28T11:38:59.737Z","pid":1993,"message":"Use Ctrl-C to stop"}1208gitlab # [ 236.845866] bundle[1993]: {"timestamp":"2026-08-28T11:38:59.738Z","pid":1993,"message":"Sending status to systemd every 1.0 sec"}1209gitlab # [ 237.585800] bundle[1993]: {"timestamp":"2026-08-28T11:39:00.479Z","pid":1993,"message":"- Worker 0 (PID: 2990) booted in 0.7s, phase: 0"}1210gitlab # [ 237.610682] bundle[1993]: {"timestamp":"2026-08-28T11:39:00.504Z","pid":1993,"message":"- Worker 1 (PID: 2994) booted in 0.7s, phase: 0"}1211gitlab # [ 237.616270] systemd[1]: Started gitlab.service.1212gitlab # [ 237.618309] systemd[1]: Reached target Common target for all GitLab services..1213gitlab # [ 237.624200] systemd[1]: Starting setup-gitlab.service...1214gitlab # [ 237.673158] setup-gitlab-start[3027]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1215gitlab # [ 241.698788] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:39:04 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 39881216gitlab # [ 241.703840] setup-gitlab-start[3027]: + TOKEN=nixbot-test-token-of-20-chars1217gitlab # [ 241.705603] setup-gitlab-start[3027]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1218gitlab # [ 241.720321] runuser[3053]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1219gitlab # [ 242.322450] setup-gitlab-start[3054]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1220gitlab # [ 242.329121] setup-gitlab-start[3054]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1221gitlab # [ 242.335408] setup-gitlab-start[3054]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1222gitlab # [ 243.213426] setup-gitlab-start[3054]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1223gitlab # [ 243.213794] setup-gitlab-start[3054]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1224gitlab # [ 243.216502] setup-gitlab-start[3054]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1225gitlab # [ 244.636401] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:07.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b97aff4247a04572fb37aa06","created_at":"2026-08-28T11:39:07.483Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91f993cd323532b15176f4fef9c762b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-28T11:39:07.524Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b97aff4247a04572fb37aa06: start","job_status":"start","queue_duration_s":0.003381,"scheduling_latency_s":0.003441}1226gitlab # [ 244.891069] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:07.782Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b97aff4247a04572fb37aa06","created_at":"2026-08-28T11:39:07.483Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91f993cd323532b15176f4fef9c762b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-28T11:39:07.524Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b97aff4247a04572fb37aa06: done: 0.254635 sec","job_status":"done","queue_duration_s":0.003381,"scheduling_latency_s":0.003441,"redis_calls":10,"redis_duration_s":0.020229999999999998,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001875,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005988,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00200252,"duration_s":0.254635,"completed_at":"2026-08-28T11:39:07.782Z","load_balancing_strategy":"primary","db_duration_s":0.063949,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1227gitlab # [ 244.991703] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:07.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"64ffc086fb9fb4e4935190c9","created_at":"2026-08-28T11:39:07.540Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.664Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-64ffc086fb9fb4e4935190c9: start","job_status":"start","queue_duration_s":0.12876,"scheduling_latency_s":0.128823}1228gitlab # [ 245.069447] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:07.961Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"64ffc086fb9fb4e4935190c9","created_at":"2026-08-28T11:39:07.540Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.664Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-64ffc086fb9fb4e4935190c9: done: 0.168771 sec","job_status":"done","queue_duration_s":0.12876,"scheduling_latency_s":0.128823,"redis_calls":19,"redis_duration_s":0.018632,"redis_read_bytes":21,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001758,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.00113,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.013644,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.0021,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00358,"duration_s":0.168771,"completed_at":"2026-08-28T11:39:07.961Z","load_balancing_strategy":"primary","db_duration_s":0.024796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1229gitlab # [ 245.155150] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:07.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5c04cffddd4fe63fd148c7ed","created_at":"2026-08-28T11:39:07.704Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60b89b01c7f27b2ca3137e5836f57289","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.736Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5c04cffddd4fe63fd148c7ed: start","job_status":"start","queue_duration_s":0.257157,"scheduling_latency_s":0.257212}1230gitlab # [ 245.275396] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.168Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1231gitlab # [ 245.317761] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5c04cffddd4fe63fd148c7ed","created_at":"2026-08-28T11:39:07.704Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60b89b01c7f27b2ca3137e5836f57289","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.736Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5c04cffddd4fe63fd148c7ed: done: 0.216598 sec","job_status":"done","queue_duration_s":0.257157,"scheduling_latency_s":0.257212,"redis_calls":10,"redis_duration_s":0.050595,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003777,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.036918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0099,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00199232,"duration_s":0.216598,"completed_at":"2026-08-28T11:39:08.209Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1232gitlab # [ 245.395381] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8d1d4b46961877c999c98139","created_at":"2026-08-28T11:39:07.770Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.791Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8d1d4b46961877c999c98139: start","job_status":"start","queue_duration_s":0.429506,"scheduling_latency_s":0.429568}1233gitlab # [ 245.897408] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.788Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8d1d4b46961877c999c98139","created_at":"2026-08-28T11:39:07.770Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.791Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8d1d4b46961877c999c98139: done: 0.568509 sec","job_status":"done","queue_duration_s":0.429506,"scheduling_latency_s":0.429568,"redis_calls":71,"redis_duration_s":0.202785,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.045972,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001655,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.152449,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002709,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.265731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050672,"duration_s":0.568509,"completed_at":"2026-08-28T11:39:08.788Z","load_balancing_strategy":"primary","db_duration_s":0.00878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1234gitlab # [ 245.992687] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ab5d0350c0b51fb850bf0fe0","created_at":"2026-08-28T11:39:07.856Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48e40be44a6ba6df990f84c9e88667ca","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.889Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ab5d0350c0b51fb850bf0fe0: start","job_status":"start","queue_duration_s":0.903029,"scheduling_latency_s":0.903066}1235gitlab # [ 246.017268] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ab5d0350c0b51fb850bf0fe0","created_at":"2026-08-28T11:39:07.856Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48e40be44a6ba6df990f84c9e88667ca","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.889Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ab5d0350c0b51fb850bf0fe0: done: 0.042884 sec","job_status":"done","queue_duration_s":0.903029,"scheduling_latency_s":0.903066,"redis_calls":12,"redis_duration_s":0.004838,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001863,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001592,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00091682,"exclusive_lock_hold_duration_s":0.0061721599999999995,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.042884,"completed_at":"2026-08-28T11:39:08.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1236gitlab # [ 246.076301] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.837Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eeb370d5080bda2f4f88f8ff","created_at":"2026-08-28T11:39:07.901Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e73d9a34fc3260b90f75a1c90ae556c0","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.923Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eeb370d5080bda2f4f88f8ff: start","job_status":"start","queue_duration_s":0.913902,"scheduling_latency_s":0.913931}1237gitlab # [ 246.091832] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.905Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eeb370d5080bda2f4f88f8ff","created_at":"2026-08-28T11:39:07.901Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e73d9a34fc3260b90f75a1c90ae556c0","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.923Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eeb370d5080bda2f4f88f8ff: done: 0.067791 sec","job_status":"done","queue_duration_s":0.913902,"scheduling_latency_s":0.913931,"redis_calls":16,"redis_duration_s":0.006258,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00413,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040198,"duration_s":0.067791,"completed_at":"2026-08-28T11:39:08.905Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020709,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1238gitlab # [ 246.135417] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.907Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cc1ec44b5ac18e80b9e16d0e","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cc1ec44b5ac18e80b9e16d0e: start","job_status":"start","queue_duration_s":0.978911,"scheduling_latency_s":0.978939}1239gitlab # [ 246.148372] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.942Z","message":"No existing merge request to be cleaned up."}1240gitlab # [ 246.150066] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.962Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cc1ec44b5ac18e80b9e16d0e","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cc1ec44b5ac18e80b9e16d0e: done: 0.055071 sec","job_status":"done","queue_duration_s":0.978911,"scheduling_latency_s":0.978939,"redis_calls":13,"redis_duration_s":0.002457,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000347,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034,"duration_s":0.055071,"completed_at":"2026-08-28T11:39:08.962Z","load_balancing_strategy":"primary","db_duration_s":0.00885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1241gitlab # [ 246.190547] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.964Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7378c90d9c24f27282e49e4a","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7378c90d9c24f27282e49e4a: start","job_status":"start","queue_duration_s":1.03579,"scheduling_latency_s":1.035817}1242gitlab # [ 246.203196] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.982Z","message":"No existing merge request to be cleaned up."}1243gitlab # [ 246.204973] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.994Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7378c90d9c24f27282e49e4a","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7378c90d9c24f27282e49e4a: done: 0.030462 sec","job_status":"done","queue_duration_s":1.03579,"scheduling_latency_s":1.035817,"redis_calls":10,"redis_duration_s":0.001794,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030462,"completed_at":"2026-08-28T11:39:08.994Z","load_balancing_strategy":"primary","db_duration_s":0.00768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1244gitlab # [ 246.244201] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:08.996Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9b7030b999e14536990785a7","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9b7030b999e14536990785a7: start","job_status":"start","queue_duration_s":1.067776,"scheduling_latency_s":1.067799}1245gitlab # [ 246.257664] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.010Z","message":"No existing merge request to be cleaned up."}1246gitlab # [ 246.259304] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.021Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9b7030b999e14536990785a7","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9b7030b999e14536990785a7: done: 0.025401 sec","job_status":"done","queue_duration_s":1.067776,"scheduling_latency_s":1.067799,"redis_calls":10,"redis_duration_s":0.001792,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025401,"completed_at":"2026-08-28T11:39:09.021Z","load_balancing_strategy":"primary","db_duration_s":0.003166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1247gitlab # [ 246.298243] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.023Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"41085409f8214625da5fd34d","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-41085409f8214625da5fd34d: start","job_status":"start","queue_duration_s":1.094972,"scheduling_latency_s":1.094994}1248gitlab # [ 246.310785] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.038Z","message":"No existing merge request to be cleaned up."}1249gitlab # [ 246.312494] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.047Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"41085409f8214625da5fd34d","created_at":"2026-08-28T11:39:07.847Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-7ea6a04864894d75","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a5146891f3128869d3173c6ac6c88c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.928Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-41085409f8214625da5fd34d: done: 0.02428 sec","job_status":"done","queue_duration_s":1.094972,"scheduling_latency_s":1.094994,"redis_calls":10,"redis_duration_s":0.0017070000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02428,"completed_at":"2026-08-28T11:39:09.047Z","load_balancing_strategy":"primary","db_duration_s":0.003017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1250gitlab # [ 246.351842] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f42c94738b6e21744a6b5cb1","created_at":"2026-08-28T11:39:07.941Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1752526e07db9fb99f145f50bc20c2ef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.968Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f42c94738b6e21744a6b5cb1: start","job_status":"start","queue_duration_s":1.080727,"scheduling_latency_s":1.080747}1251gitlab # [ 246.462771] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f42c94738b6e21744a6b5cb1","created_at":"2026-08-28T11:39:07.941Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1752526e07db9fb99f145f50bc20c2ef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:07.968Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f42c94738b6e21744a6b5cb1: done: 0.306309 sec","job_status":"done","queue_duration_s":1.080727,"scheduling_latency_s":1.080747,"redis_calls":14,"redis_duration_s":0.002771,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000578,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.063,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.223904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049442,"exclusive_lock_hold_duration_s":0.28092496,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.306309,"completed_at":"2026-08-28T11:39:09.355Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.092928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1252gitlab # [ 246.547700] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.357Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5b59f58b5067c54c91f8e5b0","created_at":"2026-08-28T11:39:07.985Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23a6d68a593649f49e366429f7325b19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.001Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5b59f58b5067c54c91f8e5b0: start","job_status":"start","queue_duration_s":1.356168,"scheduling_latency_s":1.356191}1253gitlab # [ 246.564683] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"5b59f58b5067c54c91f8e5b0","created_at":"2026-08-28T11:39:07.985Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23a6d68a593649f49e366429f7325b19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.001Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-5b59f58b5067c54c91f8e5b0: done: 0.061703 sec","job_status":"done","queue_duration_s":1.356168,"scheduling_latency_s":1.356191,"redis_calls":12,"redis_duration_s":0.00278,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007466,"exclusive_lock_hold_duration_s":0.02708286,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.061703,"completed_at":"2026-08-28T11:39:09.419Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027851,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1254gitlab # [ 246.610487] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"6f56069b612dc37aba2b2c03","created_at":"2026-08-28T11:39:08.008Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6d7ce0de7ca3e3eeb3c23bca1ea3219","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.020Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-6f56069b612dc37aba2b2c03: start","job_status":"start","queue_duration_s":1.400688,"scheduling_latency_s":1.400716}1255gitlab # [ 246.625168] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.436Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1256gitlab # [ 246.628410] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"6f56069b612dc37aba2b2c03","created_at":"2026-08-28T11:39:08.008Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6d7ce0de7ca3e3eeb3c23bca1ea3219","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.020Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-6f56069b612dc37aba2b2c03: done: 0.024866 sec","job_status":"done","queue_duration_s":1.400688,"scheduling_latency_s":1.400716,"redis_calls":10,"redis_duration_s":0.002407,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033422,"duration_s":0.024866,"completed_at":"2026-08-28T11:39:09.445Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1257gitlab # [ 246.672854] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.447Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d7f06010d6dc6f93babf71f5","created_at":"2026-08-28T11:39:08.054Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cb0ff1f2e40b50a38cdc4407a72cad1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.062Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d7f06010d6dc6f93babf71f5: start","job_status":"start","queue_duration_s":1.385315,"scheduling_latency_s":1.385338}1258gitlab # [ 246.686904] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d7f06010d6dc6f93babf71f5","created_at":"2026-08-28T11:39:08.054Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cb0ff1f2e40b50a38cdc4407a72cad1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.062Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d7f06010d6dc6f93babf71f5: done: 0.054842 sec","job_status":"done","queue_duration_s":1.385315,"scheduling_latency_s":1.385338,"redis_calls":10,"redis_duration_s":0.002221,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003634,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.054842,"completed_at":"2026-08-28T11:39:09.502Z","load_balancing_strategy":"primary","db_duration_s":0.026302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1259gitlab # [ 246.730994] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"495c9695052af29f14af1add","created_at":"2026-08-28T11:39:08.079Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"528c428802d79adc6a3045b683ab6e98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.086Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-495c9695052af29f14af1add: start","job_status":"start","queue_duration_s":1.417573,"scheduling_latency_s":1.417598}1260gitlab # [ 246.745664] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.521Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1261gitlab # [ 246.748827] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.530Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"495c9695052af29f14af1add","created_at":"2026-08-28T11:39:08.079Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"528c428802d79adc6a3045b683ab6e98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.086Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-495c9695052af29f14af1add: done: 0.026504 sec","job_status":"done","queue_duration_s":1.417573,"scheduling_latency_s":1.417598,"redis_calls":10,"redis_duration_s":0.002203,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000492,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003095,"duration_s":0.026504,"completed_at":"2026-08-28T11:39:09.530Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1262gitlab # [ 246.791943] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bdee32b0e985fc892b9f0bb7","created_at":"2026-08-28T11:39:08.102Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"912bb047043ebe2eaf01c49e25dcfa46","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.124Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bdee32b0e985fc892b9f0bb7: start","job_status":"start","queue_duration_s":1.408609,"scheduling_latency_s":1.408631}1263gitlab # [ 246.806899] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bdee32b0e985fc892b9f0bb7","created_at":"2026-08-28T11:39:08.102Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"912bb047043ebe2eaf01c49e25dcfa46","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.124Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bdee32b0e985fc892b9f0bb7: done: 0.057729 sec","job_status":"done","queue_duration_s":1.408609,"scheduling_latency_s":1.408631,"redis_calls":12,"redis_duration_s":0.0026060000000000002,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00064162,"exclusive_lock_hold_duration_s":0.032201099999999996,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.057729,"completed_at":"2026-08-28T11:39:09.590Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1264gitlab # [ 246.852661] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fb4324f2f7ad7b330fa34cba","created_at":"2026-08-28T11:39:08.150Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f64d438079a4d49a3080cd3e20df747","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.170Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fb4324f2f7ad7b330fa34cba: start","job_status":"start","queue_duration_s":1.42168,"scheduling_latency_s":1.421703}1265gitlab # [ 246.868417] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fb4324f2f7ad7b330fa34cba","created_at":"2026-08-28T11:39:08.150Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f64d438079a4d49a3080cd3e20df747","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.170Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fb4324f2f7ad7b330fa34cba: done: 0.025536 sec","job_status":"done","queue_duration_s":1.42168,"scheduling_latency_s":1.421703,"redis_calls":12,"redis_duration_s":0.0018340000000000001,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000625,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000356,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026124,"duration_s":0.025536,"completed_at":"2026-08-28T11:39:09.617Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1266gitlab # [ 246.912250] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"06f027ed7fcb3a5c4cfb41c4","created_at":"2026-08-28T11:39:08.189Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"999fb39854a8d5571505b2311daa53dd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.217Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-06f027ed7fcb3a5c4cfb41c4: start","job_status":"start","queue_duration_s":1.401855,"scheduling_latency_s":1.401875}1267gitlab # [ 246.926069] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"06f027ed7fcb3a5c4cfb41c4","created_at":"2026-08-28T11:39:08.189Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"999fb39854a8d5571505b2311daa53dd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.217Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-06f027ed7fcb3a5c4cfb41c4: done: 0.03057 sec","job_status":"done","queue_duration_s":1.401855,"scheduling_latency_s":1.401875,"redis_calls":12,"redis_duration_s":0.002049,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007126,"exclusive_lock_hold_duration_s":0.00980786,"duration_s":0.03057,"completed_at":"2026-08-28T11:39:09.649Z","load_balancing_strategy":"primary","db_duration_s":0.005505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1268gitlab # [ 246.968106] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ab7c286ff414edeb5c6efa95","created_at":"2026-08-28T11:39:08.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78c059a5dc3dae6c85d2bbc7dd99df5b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.273Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ab7c286ff414edeb5c6efa95: start","job_status":"start","queue_duration_s":1.378127,"scheduling_latency_s":1.378145}1269gitlab # [ 246.982227] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ab7c286ff414edeb5c6efa95","created_at":"2026-08-28T11:39:08.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78c059a5dc3dae6c85d2bbc7dd99df5b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.273Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ab7c286ff414edeb5c6efa95: done: 0.027385 sec","job_status":"done","queue_duration_s":1.378127,"scheduling_latency_s":1.378145,"redis_calls":10,"redis_duration_s":0.001572,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023928,"duration_s":0.027385,"completed_at":"2026-08-28T11:39:09.678Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 247.024635] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.680Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"5a7151bd9139954a73e919c8","created_at":"2026-08-28T11:39:08.308Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4398c3b2ec5cb83f4b12a3afc8b0465c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.330Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-5a7151bd9139954a73e919c8: start","job_status":"start","queue_duration_s":1.349847,"scheduling_latency_s":1.349866}1271gitlab # [ 247.047634] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"5a7151bd9139954a73e919c8","created_at":"2026-08-28T11:39:08.308Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4398c3b2ec5cb83f4b12a3afc8b0465c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.330Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-5a7151bd9139954a73e919c8: done: 0.03509 sec","job_status":"done","queue_duration_s":1.349847,"scheduling_latency_s":1.349866,"redis_calls":13,"redis_duration_s":0.002127,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000749,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01117828,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.03509,"completed_at":"2026-08-28T11:39:09.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004271,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1272gitlab # [ 247.106105] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"56588910bc32775bf02694b7","created_at":"2026-08-28T11:39:08.378Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8b507b09b327dfb56e5475ff925fe3a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-28T11:39:08.411Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-56588910bc32775bf02694b7: start","job_status":"start","queue_duration_s":1.305229,"scheduling_latency_s":1.305249}1273gitlab # [ 247.120796] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.728Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1274gitlab # [ 247.124130] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.736Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"56588910bc32775bf02694b7","created_at":"2026-08-28T11:39:08.378Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8b507b09b327dfb56e5475ff925fe3a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-28T11:39:08.411Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-56588910bc32775bf02694b7: done: 0.019608 sec","job_status":"done","queue_duration_s":1.305229,"scheduling_latency_s":1.305249,"redis_calls":10,"redis_duration_s":0.001354,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024256,"duration_s":0.019608,"completed_at":"2026-08-28T11:39:09.736Z","load_balancing_strategy":"primary","db_duration_s":0.000657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1275gitlab # [ 247.165058] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"192fd1d7024d5e53d0e6db1c","created_at":"2026-08-28T11:39:08.435Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c328e43eb46d3d449a6c364df14e92f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.463Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-192fd1d7024d5e53d0e6db1c: start","job_status":"start","queue_duration_s":1.274401,"scheduling_latency_s":1.274418}1276gitlab # [ 247.179376] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.749Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1277gitlab # [ 247.183109] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"192fd1d7024d5e53d0e6db1c","created_at":"2026-08-28T11:39:08.435Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c328e43eb46d3d449a6c364df14e92f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.463Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-192fd1d7024d5e53d0e6db1c: done: 0.020007 sec","job_status":"done","queue_duration_s":1.274401,"scheduling_latency_s":1.274418,"redis_calls":10,"redis_duration_s":0.001432,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047478,"duration_s":0.020007,"completed_at":"2026-08-28T11:39:09.757Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1278gitlab # [ 247.225063] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e3d0cb5351916ddf95bfbcef","created_at":"2026-08-28T11:39:08.490Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5767ec52f0885a2f6dfd1aeb9b2177b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-28T11:39:08.520Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e3d0cb5351916ddf95bfbcef: start","job_status":"start","queue_duration_s":1.238871,"scheduling_latency_s":1.23889}1279gitlab # [ 247.238322] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.771Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1280gitlab # [ 247.242118] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.779Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e3d0cb5351916ddf95bfbcef","created_at":"2026-08-28T11:39:08.490Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5767ec52f0885a2f6dfd1aeb9b2177b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-28T11:39:08.520Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e3d0cb5351916ddf95bfbcef: done: 0.020006 sec","job_status":"done","queue_duration_s":1.238871,"scheduling_latency_s":1.23889,"redis_calls":10,"redis_duration_s":0.001501,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000345,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029556,"duration_s":0.020006,"completed_at":"2026-08-28T11:39:09.779Z","load_balancing_strategy":"primary","db_duration_s":0.000741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1281gitlab # [ 247.284218] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.780Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cbb8c6edeb1b0a5a1c8d05bb","created_at":"2026-08-28T11:39:08.567Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"932ed5133e2137939d272f84cfa16041","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.593Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cbb8c6edeb1b0a5a1c8d05bb: start","job_status":"start","queue_duration_s":1.18684,"scheduling_latency_s":1.18686}1282gitlab # [ 247.300775] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.822Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cbb8c6edeb1b0a5a1c8d05bb","created_at":"2026-08-28T11:39:08.567Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"932ed5133e2137939d272f84cfa16041","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.593Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cbb8c6edeb1b0a5a1c8d05bb: done: 0.042059 sec","job_status":"done","queue_duration_s":1.18684,"scheduling_latency_s":1.18686,"redis_calls":10,"redis_duration_s":0.0035719999999999997,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002351,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000764,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003441,"duration_s":0.042059,"completed_at":"2026-08-28T11:39:09.822Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1283gitlab # [ 247.343257] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:09.824Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1e93f67ab31d824ece549e97","created_at":"2026-08-28T11:39:08.634Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a22f2ea96f752ab84c7d91b6653b3ab4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.662Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1e93f67ab31d824ece549e97: start","job_status":"start","queue_duration_s":1.162277,"scheduling_latency_s":1.162309}1284gitlab # [ 247.456969] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.350Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1e93f67ab31d824ece549e97","created_at":"2026-08-28T11:39:08.634Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a22f2ea96f752ab84c7d91b6653b3ab4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.662Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1e93f67ab31d824ece549e97: done: 0.525781 sec","job_status":"done","queue_duration_s":1.162277,"scheduling_latency_s":1.162309,"redis_calls":565,"redis_duration_s":0.08954100000000001,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.087911,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.477453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037018,"duration_s":0.525781,"completed_at":"2026-08-28T11:39:10.350Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1285gitlab # [ 247.537670] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"853d135701db54060c491c51","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-853d135701db54060c491c51: start","job_status":"start","queue_duration_s":1.579717,"scheduling_latency_s":1.57974}1286gitlab # [ 247.552914] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"853d135701db54060c491c51","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-853d135701db54060c491c51: done: 0.026415 sec","job_status":"done","queue_duration_s":1.579717,"scheduling_latency_s":1.57974,"redis_calls":16,"redis_duration_s":0.00395,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000578,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001177,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002195,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034136,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026415,"completed_at":"2026-08-28T11:39:10.378Z","load_balancing_strategy":"primary","db_duration_s":0.001055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1287gitlab # [ 247.594934] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d2628bbeea7cc84c8d32864","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d2628bbeea7cc84c8d32864: start","job_status":"start","queue_duration_s":1.607881,"scheduling_latency_s":1.607903}1288gitlab # [ 247.607471] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d2628bbeea7cc84c8d32864","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d2628bbeea7cc84c8d32864: done: 0.017996 sec","job_status":"done","queue_duration_s":1.607881,"scheduling_latency_s":1.607903,"redis_calls":13,"redis_duration_s":0.002951,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017996,"completed_at":"2026-08-28T11:39:10.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1289gitlab # [ 247.646757] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"516d77195008eb2653576cf0","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-516d77195008eb2653576cf0: start","job_status":"start","queue_duration_s":1.627552,"scheduling_latency_s":1.627572}1290gitlab # [ 247.659389] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"516d77195008eb2653576cf0","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-516d77195008eb2653576cf0: done: 0.019004 sec","job_status":"done","queue_duration_s":1.627552,"scheduling_latency_s":1.627572,"redis_calls":13,"redis_duration_s":0.0031929999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019004,"completed_at":"2026-08-28T11:39:10.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 247.700321] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a04912e9d363e97067b07dbf","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a04912e9d363e97067b07dbf: start","job_status":"start","queue_duration_s":1.649162,"scheduling_latency_s":1.649183}1292gitlab # [ 247.712644] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a04912e9d363e97067b07dbf","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a04912e9d363e97067b07dbf: done: 0.018504 sec","job_status":"done","queue_duration_s":1.649162,"scheduling_latency_s":1.649183,"redis_calls":13,"redis_duration_s":0.003149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018504,"completed_at":"2026-08-28T11:39:10.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1293gitlab # [ 247.752409] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c6326902e22a8b02dbf77d7","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c6326902e22a8b02dbf77d7: start","job_status":"start","queue_duration_s":1.669325,"scheduling_latency_s":1.669346}1294gitlab # [ 247.764749] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c6326902e22a8b02dbf77d7","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c6326902e22a8b02dbf77d7: done: 0.016681 sec","job_status":"done","queue_duration_s":1.669325,"scheduling_latency_s":1.669346,"redis_calls":13,"redis_duration_s":0.0028729999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016681,"completed_at":"2026-08-28T11:39:10.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1295gitlab # [ 247.804344] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce3c8d847aceac99deb7178d","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce3c8d847aceac99deb7178d: start","job_status":"start","queue_duration_s":1.687452,"scheduling_latency_s":1.68747}1296gitlab # [ 247.817410] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce3c8d847aceac99deb7178d","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce3c8d847aceac99deb7178d: done: 0.015911 sec","job_status":"done","queue_duration_s":1.687452,"scheduling_latency_s":1.68747,"redis_calls":13,"redis_duration_s":0.002723,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001781,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015911,"completed_at":"2026-08-28T11:39:10.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1297gitlab # [ 247.903164] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ba3bfc3309581c4e9888ca","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ba3bfc3309581c4e9888ca: start","job_status":"start","queue_duration_s":1.704801,"scheduling_latency_s":1.704817}1298gitlab # [ 247.921016] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ba3bfc3309581c4e9888ca","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ba3bfc3309581c4e9888ca: done: 0.017511 sec","job_status":"done","queue_duration_s":1.704801,"scheduling_latency_s":1.704817,"redis_calls":13,"redis_duration_s":0.0034029999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00242,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017511,"completed_at":"2026-08-28T11:39:10.494Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1299gitlab # [ 247.970135] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cc47ef1a4fb72d3b2986d96","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cc47ef1a4fb72d3b2986d96: start","job_status":"start","queue_duration_s":1.723959,"scheduling_latency_s":1.723975}1300gitlab # [ 247.983301] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cc47ef1a4fb72d3b2986d96","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cc47ef1a4fb72d3b2986d96: done: 0.016443 sec","job_status":"done","queue_duration_s":1.723959,"scheduling_latency_s":1.723975,"redis_calls":13,"redis_duration_s":0.002929,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016443,"completed_at":"2026-08-28T11:39:10.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 248.022774] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62d50e81229c10abacbdeb61","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62d50e81229c10abacbdeb61: start","job_status":"start","queue_duration_s":1.741987,"scheduling_latency_s":1.742004}1302gitlab # [ 248.035345] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62d50e81229c10abacbdeb61","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62d50e81229c10abacbdeb61: done: 0.017141 sec","job_status":"done","queue_duration_s":1.741987,"scheduling_latency_s":1.742004,"redis_calls":13,"redis_duration_s":0.0036039999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002541,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017141,"completed_at":"2026-08-28T11:39:10.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 248.074468] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fa8b9134a8ed38f43134a79","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fa8b9134a8ed38f43134a79: start","job_status":"start","queue_duration_s":1.760623,"scheduling_latency_s":1.760641}1304gitlab # [ 248.086938] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fa8b9134a8ed38f43134a79","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fa8b9134a8ed38f43134a79: done: 0.016353 sec","job_status":"done","queue_duration_s":1.760623,"scheduling_latency_s":1.760641,"redis_calls":13,"redis_duration_s":0.0029259999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016353,"completed_at":"2026-08-28T11:39:10.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 248.127537] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e621bae1f211642f5ffe820","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e621bae1f211642f5ffe820: start","job_status":"start","queue_duration_s":1.779368,"scheduling_latency_s":1.779387}1306gitlab # [ 248.140186] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e621bae1f211642f5ffe820","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e621bae1f211642f5ffe820: done: 0.015969 sec","job_status":"done","queue_duration_s":1.779368,"scheduling_latency_s":1.779387,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001823,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015969,"completed_at":"2026-08-28T11:39:10.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 248.179621] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e09212f6bc5f77c661116d8e","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e09212f6bc5f77c661116d8e: start","job_status":"start","queue_duration_s":1.796718,"scheduling_latency_s":1.796736}1308gitlab # [ 248.192158] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e09212f6bc5f77c661116d8e","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e09212f6bc5f77c661116d8e: done: 0.01586 sec","job_status":"done","queue_duration_s":1.796718,"scheduling_latency_s":1.796736,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01586,"completed_at":"2026-08-28T11:39:10.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 248.231351] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"971956564090d3119f6eb7dd","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-971956564090d3119f6eb7dd: start","job_status":"start","queue_duration_s":1.813991,"scheduling_latency_s":1.814009}1310gitlab # [ 248.243902] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"971956564090d3119f6eb7dd","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-971956564090d3119f6eb7dd: done: 0.016064 sec","job_status":"done","queue_duration_s":1.813991,"scheduling_latency_s":1.814009,"redis_calls":13,"redis_duration_s":0.002748,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001808,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016064,"completed_at":"2026-08-28T11:39:10.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 248.283328] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d792f0200a8ef919f13af40","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d792f0200a8ef919f13af40: start","job_status":"start","queue_duration_s":1.831528,"scheduling_latency_s":1.831545}1312gitlab # [ 248.295874] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d792f0200a8ef919f13af40","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d792f0200a8ef919f13af40: done: 0.01645 sec","job_status":"done","queue_duration_s":1.831528,"scheduling_latency_s":1.831545,"redis_calls":13,"redis_duration_s":0.002534,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001567,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01645,"completed_at":"2026-08-28T11:39:10.620Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 248.336236] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b43b8968eb8939305e346c2","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b43b8968eb8939305e346c2: start","job_status":"start","queue_duration_s":1.849429,"scheduling_latency_s":1.849446}1314gitlab # [ 248.348580] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b43b8968eb8939305e346c2","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b43b8968eb8939305e346c2: done: 0.015894 sec","job_status":"done","queue_duration_s":1.849429,"scheduling_latency_s":1.849446,"redis_calls":13,"redis_duration_s":0.0027099999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015894,"completed_at":"2026-08-28T11:39:10.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 248.388125] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f140e63259bcb2c8cee642ef","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f140e63259bcb2c8cee642ef: start","job_status":"start","queue_duration_s":1.866696,"scheduling_latency_s":1.866714}1316gitlab # [ 248.400492] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f140e63259bcb2c8cee642ef","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f140e63259bcb2c8cee642ef: done: 0.015961 sec","job_status":"done","queue_duration_s":1.866696,"scheduling_latency_s":1.866714,"redis_calls":13,"redis_duration_s":0.002712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015961,"completed_at":"2026-08-28T11:39:10.654Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 248.439690] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c938193aaacbe1c725f4223","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c938193aaacbe1c725f4223: start","job_status":"start","queue_duration_s":1.884134,"scheduling_latency_s":1.884153}1318gitlab # [ 248.452144] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c938193aaacbe1c725f4223","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c938193aaacbe1c725f4223: done: 0.01745 sec","job_status":"done","queue_duration_s":1.884134,"scheduling_latency_s":1.884153,"redis_calls":13,"redis_duration_s":0.004119,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01745,"completed_at":"2026-08-28T11:39:10.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 248.491225] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc9aa8c32f36b99a9e9ee2a6","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc9aa8c32f36b99a9e9ee2a6: start","job_status":"start","queue_duration_s":1.90413,"scheduling_latency_s":1.904151}1320gitlab # [ 248.503762] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc9aa8c32f36b99a9e9ee2a6","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc9aa8c32f36b99a9e9ee2a6: done: 0.015906 sec","job_status":"done","queue_duration_s":1.90413,"scheduling_latency_s":1.904151,"redis_calls":13,"redis_duration_s":0.003043,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001489,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015906,"completed_at":"2026-08-28T11:39:10.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 248.545022] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"579b51d90b539b62da71f24e","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-579b51d90b539b62da71f24e: start","job_status":"start","queue_duration_s":1.921301,"scheduling_latency_s":1.921317}1322gitlab # [ 248.582497] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"579b51d90b539b62da71f24e","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-579b51d90b539b62da71f24e: done: 0.015295 sec","job_status":"done","queue_duration_s":1.921301,"scheduling_latency_s":1.921317,"redis_calls":13,"redis_duration_s":0.0025830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001671,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015295,"completed_at":"2026-08-28T11:39:10.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 248.651365] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53c7a6683f511369269be03c","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53c7a6683f511369269be03c: start","job_status":"start","queue_duration_s":1.937973,"scheduling_latency_s":1.93799}1324gitlab # [ 248.665738] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53c7a6683f511369269be03c","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53c7a6683f511369269be03c: done: 0.01679 sec","job_status":"done","queue_duration_s":1.937973,"scheduling_latency_s":1.93799,"redis_calls":13,"redis_duration_s":0.001954,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001369,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01679,"completed_at":"2026-08-28T11:39:10.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 248.705706] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9e7b21a682f82ea1fde1da4","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9e7b21a682f82ea1fde1da4: start","job_status":"start","queue_duration_s":1.955962,"scheduling_latency_s":1.955979}1326gitlab # [ 248.718238] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9e7b21a682f82ea1fde1da4","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9e7b21a682f82ea1fde1da4: done: 0.014578 sec","job_status":"done","queue_duration_s":1.955962,"scheduling_latency_s":1.955979,"redis_calls":13,"redis_duration_s":0.001667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014578,"completed_at":"2026-08-28T11:39:10.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 248.760746] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a422f9ef5e93ed2ade07ee","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a422f9ef5e93ed2ade07ee: start","job_status":"start","queue_duration_s":1.971737,"scheduling_latency_s":1.971753}1328gitlab # [ 248.773892] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a422f9ef5e93ed2ade07ee","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a422f9ef5e93ed2ade07ee: done: 0.015208 sec","job_status":"done","queue_duration_s":1.971737,"scheduling_latency_s":1.971753,"redis_calls":13,"redis_duration_s":0.0018709999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000679,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015208,"completed_at":"2026-08-28T11:39:10.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 248.814354] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86dd1c2d5bb2492aef7651b9","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86dd1c2d5bb2492aef7651b9: start","job_status":"start","queue_duration_s":1.988182,"scheduling_latency_s":1.988197}1330gitlab # [ 248.827088] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86dd1c2d5bb2492aef7651b9","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86dd1c2d5bb2492aef7651b9: done: 0.016208 sec","job_status":"done","queue_duration_s":1.988182,"scheduling_latency_s":1.988197,"redis_calls":13,"redis_duration_s":0.0026950000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001435,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016208,"completed_at":"2026-08-28T11:39:10.776Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 248.867045] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dfe0bc0d553253b3cc35c0b","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dfe0bc0d553253b3cc35c0b: start","job_status":"start","queue_duration_s":2.005761,"scheduling_latency_s":2.005779}1332gitlab # [ 248.879755] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dfe0bc0d553253b3cc35c0b","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dfe0bc0d553253b3cc35c0b: done: 0.016399 sec","job_status":"done","queue_duration_s":2.005761,"scheduling_latency_s":2.005779,"redis_calls":13,"redis_duration_s":0.0025859999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001678,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016399,"completed_at":"2026-08-28T11:39:10.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 248.919580] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"482f1a6785fcb128da987a0f","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-482f1a6785fcb128da987a0f: start","job_status":"start","queue_duration_s":2.023578,"scheduling_latency_s":2.023595}1334gitlab # [ 248.933862] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"482f1a6785fcb128da987a0f","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-482f1a6785fcb128da987a0f: done: 0.015501 sec","job_status":"done","queue_duration_s":2.023578,"scheduling_latency_s":2.023595,"redis_calls":13,"redis_duration_s":0.002845,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015501,"completed_at":"2026-08-28T11:39:10.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 248.973386] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b55d0ed8b1867557e28fd72","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b55d0ed8b1867557e28fd72: start","job_status":"start","queue_duration_s":2.040514,"scheduling_latency_s":2.040531}1336gitlab # [ 248.985774] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b55d0ed8b1867557e28fd72","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b55d0ed8b1867557e28fd72: done: 0.015656 sec","job_status":"done","queue_duration_s":2.040514,"scheduling_latency_s":2.040531,"redis_calls":13,"redis_duration_s":0.0032689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002355,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015656,"completed_at":"2026-08-28T11:39:10.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 249.025228] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"452f3e27213aacd23f7f8ac8","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-452f3e27213aacd23f7f8ac8: start","job_status":"start","queue_duration_s":2.057591,"scheduling_latency_s":2.057607}1338gitlab # [ 249.037740] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"452f3e27213aacd23f7f8ac8","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-452f3e27213aacd23f7f8ac8: done: 0.014899 sec","job_status":"done","queue_duration_s":2.057591,"scheduling_latency_s":2.057607,"redis_calls":13,"redis_duration_s":0.0027159999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014899,"completed_at":"2026-08-28T11:39:10.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 249.077212] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1911dba80153eca1efcac0ec","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1911dba80153eca1efcac0ec: start","job_status":"start","queue_duration_s":2.07395,"scheduling_latency_s":2.073967}1340gitlab # [ 249.089671] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1911dba80153eca1efcac0ec","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1911dba80153eca1efcac0ec: done: 0.015331 sec","job_status":"done","queue_duration_s":2.07395,"scheduling_latency_s":2.073967,"redis_calls":13,"redis_duration_s":0.0029300000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015331,"completed_at":"2026-08-28T11:39:10.861Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 249.130226] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acb95675e902bdbfba50f7bc","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acb95675e902bdbfba50f7bc: start","job_status":"start","queue_duration_s":2.090712,"scheduling_latency_s":2.09073}1342gitlab # [ 249.143176] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acb95675e902bdbfba50f7bc","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acb95675e902bdbfba50f7bc: done: 0.014893 sec","job_status":"done","queue_duration_s":2.090712,"scheduling_latency_s":2.09073,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014893,"completed_at":"2026-08-28T11:39:10.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 249.182836] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"983f9c895d42c20d0d1c9b2b","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-983f9c895d42c20d0d1c9b2b: start","job_status":"start","queue_duration_s":2.107293,"scheduling_latency_s":2.107311}1344gitlab # [ 249.195799] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"983f9c895d42c20d0d1c9b2b","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-983f9c895d42c20d0d1c9b2b: done: 0.015054 sec","job_status":"done","queue_duration_s":2.107293,"scheduling_latency_s":2.107311,"redis_calls":13,"redis_duration_s":0.002834,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015054,"completed_at":"2026-08-28T11:39:10.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 249.235794] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16ae21dc3040bc6d55e5d53a","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16ae21dc3040bc6d55e5d53a: start","job_status":"start","queue_duration_s":2.123769,"scheduling_latency_s":2.123785}1346gitlab # [ 249.248575] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16ae21dc3040bc6d55e5d53a","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16ae21dc3040bc6d55e5d53a: done: 0.015775 sec","job_status":"done","queue_duration_s":2.123769,"scheduling_latency_s":2.123785,"redis_calls":13,"redis_duration_s":0.0029040000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015775,"completed_at":"2026-08-28T11:39:10.911Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 249.288149] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bdfa48b01e77a21ff8a8bb7","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bdfa48b01e77a21ff8a8bb7: start","job_status":"start","queue_duration_s":2.140996,"scheduling_latency_s":2.141014}1348gitlab # [ 249.300531] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bdfa48b01e77a21ff8a8bb7","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bdfa48b01e77a21ff8a8bb7: done: 0.016099 sec","job_status":"done","queue_duration_s":2.140996,"scheduling_latency_s":2.141014,"redis_calls":13,"redis_duration_s":0.0033120000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002341,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016099,"completed_at":"2026-08-28T11:39:10.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 249.341470] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f19870f471fa58d8e636fe7","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f19870f471fa58d8e636fe7: start","job_status":"start","queue_duration_s":2.158622,"scheduling_latency_s":2.158638}1350gitlab # [ 249.354014] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f19870f471fa58d8e636fe7","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f19870f471fa58d8e636fe7: done: 0.015427 sec","job_status":"done","queue_duration_s":2.158622,"scheduling_latency_s":2.158638,"redis_calls":13,"redis_duration_s":0.002809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001841,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015427,"completed_at":"2026-08-28T11:39:10.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 249.393254] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"288ccd3acf3d2e9a5141d213","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-288ccd3acf3d2e9a5141d213: start","job_status":"start","queue_duration_s":2.175627,"scheduling_latency_s":2.175643}1352gitlab # [ 249.405691] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"288ccd3acf3d2e9a5141d213","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-288ccd3acf3d2e9a5141d213: done: 0.014786 sec","job_status":"done","queue_duration_s":2.175627,"scheduling_latency_s":2.175643,"redis_calls":13,"redis_duration_s":0.0028980000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014786,"completed_at":"2026-08-28T11:39:10.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 249.445645] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b368ada5acaf41d4770c660","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b368ada5acaf41d4770c660: start","job_status":"start","queue_duration_s":2.191804,"scheduling_latency_s":2.19182}1354gitlab # [ 249.458249] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b368ada5acaf41d4770c660","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b368ada5acaf41d4770c660: done: 0.01511 sec","job_status":"done","queue_duration_s":2.191804,"scheduling_latency_s":2.19182,"redis_calls":13,"redis_duration_s":0.0027170000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001783,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01511,"completed_at":"2026-08-28T11:39:10.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 249.497449] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:10.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3954c1b34300bf8cc5fa7fa","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3954c1b34300bf8cc5fa7fa: start","job_status":"start","queue_duration_s":2.208373,"scheduling_latency_s":2.208389}1356gitlab # [ 249.509976] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3954c1b34300bf8cc5fa7fa","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3954c1b34300bf8cc5fa7fa: done: 0.025198 sec","job_status":"done","queue_duration_s":2.208373,"scheduling_latency_s":2.208389,"redis_calls":13,"redis_duration_s":0.010855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006339,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025198,"completed_at":"2026-08-28T11:39:11.005Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 249.550491] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f12b9027491185abdd53c22","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f12b9027491185abdd53c22: start","job_status":"start","queue_duration_s":2.235645,"scheduling_latency_s":2.235665}1358gitlab # [ 249.563095] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f12b9027491185abdd53c22","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f12b9027491185abdd53c22: done: 0.01658 sec","job_status":"done","queue_duration_s":2.235645,"scheduling_latency_s":2.235665,"redis_calls":13,"redis_duration_s":0.001987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001302,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01658,"completed_at":"2026-08-28T11:39:11.024Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 249.602616] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c5f47a2c162e5e4acf94634","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c5f47a2c162e5e4acf94634: start","job_status":"start","queue_duration_s":2.253596,"scheduling_latency_s":2.253615}1360gitlab # [ 249.615084] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c5f47a2c162e5e4acf94634","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c5f47a2c162e5e4acf94634: done: 0.016044 sec","job_status":"done","queue_duration_s":2.253596,"scheduling_latency_s":2.253615,"redis_calls":13,"redis_duration_s":0.002467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001819,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016044,"completed_at":"2026-08-28T11:39:11.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 249.654555] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d21e2496ab818969459dc439","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d21e2496ab818969459dc439: start","job_status":"start","queue_duration_s":2.271047,"scheduling_latency_s":2.271067}1362gitlab # [ 249.667196] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d21e2496ab818969459dc439","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d21e2496ab818969459dc439: done: 0.015028 sec","job_status":"done","queue_duration_s":2.271047,"scheduling_latency_s":2.271067,"redis_calls":13,"redis_duration_s":0.0017720000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015028,"completed_at":"2026-08-28T11:39:11.058Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 249.706851] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f569564a2b93e6344b71b1","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f569564a2b93e6344b71b1: start","job_status":"start","queue_duration_s":2.287662,"scheduling_latency_s":2.287681}1364gitlab # [ 249.719180] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f569564a2b93e6344b71b1","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f569564a2b93e6344b71b1: done: 0.014166 sec","job_status":"done","queue_duration_s":2.287662,"scheduling_latency_s":2.287681,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014166,"completed_at":"2026-08-28T11:39:11.074Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 249.759989] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9122407d1788a0aeb70c772","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9122407d1788a0aeb70c772: start","job_status":"start","queue_duration_s":2.302975,"scheduling_latency_s":2.302992}1366gitlab # [ 249.772337] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9122407d1788a0aeb70c772","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9122407d1788a0aeb70c772: done: 0.014153 sec","job_status":"done","queue_duration_s":2.302975,"scheduling_latency_s":2.302992,"redis_calls":13,"redis_duration_s":0.0014990000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014153,"completed_at":"2026-08-28T11:39:11.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 249.812040] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d99c6ab58e510b8255bb458","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d99c6ab58e510b8255bb458: start","job_status":"start","queue_duration_s":2.318288,"scheduling_latency_s":2.318306}1368gitlab # [ 249.824280] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d99c6ab58e510b8255bb458","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d99c6ab58e510b8255bb458: done: 0.014834 sec","job_status":"done","queue_duration_s":2.318288,"scheduling_latency_s":2.318306,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014834,"completed_at":"2026-08-28T11:39:11.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 249.863331] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45616ec1dd8d6e4d9ea07c08","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45616ec1dd8d6e4d9ea07c08: start","job_status":"start","queue_duration_s":2.334298,"scheduling_latency_s":2.334315}1370gitlab # [ 249.875862] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45616ec1dd8d6e4d9ea07c08","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45616ec1dd8d6e4d9ea07c08: done: 0.014405 sec","job_status":"done","queue_duration_s":2.334298,"scheduling_latency_s":2.334315,"redis_calls":13,"redis_duration_s":0.001471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014405,"completed_at":"2026-08-28T11:39:11.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 249.915246] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c68d09f0eb3b0c83ea80984f","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c68d09f0eb3b0c83ea80984f: start","job_status":"start","queue_duration_s":2.349881,"scheduling_latency_s":2.349898}1372gitlab # [ 249.927729] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c68d09f0eb3b0c83ea80984f","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c68d09f0eb3b0c83ea80984f: done: 0.014935 sec","job_status":"done","queue_duration_s":2.349881,"scheduling_latency_s":2.349898,"redis_calls":13,"redis_duration_s":0.0018210000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014935,"completed_at":"2026-08-28T11:39:11.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 249.968520] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"732ca7feb8ecc62ec66461ec","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-732ca7feb8ecc62ec66461ec: start","job_status":"start","queue_duration_s":2.366029,"scheduling_latency_s":2.366046}1374gitlab # [ 249.981004] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"732ca7feb8ecc62ec66461ec","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-732ca7feb8ecc62ec66461ec: done: 0.01478 sec","job_status":"done","queue_duration_s":2.366029,"scheduling_latency_s":2.366046,"redis_calls":13,"redis_duration_s":0.001684,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01478,"completed_at":"2026-08-28T11:39:11.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 250.020449] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60a6ae0c90d5dfb3b7702755","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60a6ae0c90d5dfb3b7702755: start","job_status":"start","queue_duration_s":2.38202,"scheduling_latency_s":2.382039}1376gitlab # [ 250.032699] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60a6ae0c90d5dfb3b7702755","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60a6ae0c90d5dfb3b7702755: done: 0.014646 sec","job_status":"done","queue_duration_s":2.38202,"scheduling_latency_s":2.382039,"redis_calls":13,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014646,"completed_at":"2026-08-28T11:39:11.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 250.071930] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"083fcb9959ef321d52d6d19c","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-083fcb9959ef321d52d6d19c: start","job_status":"start","queue_duration_s":2.398563,"scheduling_latency_s":2.398581}1378gitlab # [ 250.084593] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"083fcb9959ef321d52d6d19c","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-083fcb9959ef321d52d6d19c: done: 0.015982 sec","job_status":"done","queue_duration_s":2.398563,"scheduling_latency_s":2.398581,"redis_calls":13,"redis_duration_s":0.002517,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015982,"completed_at":"2026-08-28T11:39:11.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 250.123992] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f22469df1180438e36589a","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f22469df1180438e36589a: start","job_status":"start","queue_duration_s":2.416516,"scheduling_latency_s":2.416536}1380gitlab # [ 250.136161] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53f22469df1180438e36589a","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53f22469df1180438e36589a: done: 0.015723 sec","job_status":"done","queue_duration_s":2.416516,"scheduling_latency_s":2.416536,"redis_calls":13,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015723,"completed_at":"2026-08-28T11:39:11.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 250.176765] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"206944fd40eb197cbbd722ab","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-206944fd40eb197cbbd722ab: start","job_status":"start","queue_duration_s":2.433427,"scheduling_latency_s":2.433445}1382gitlab # [ 250.189176] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"206944fd40eb197cbbd722ab","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-206944fd40eb197cbbd722ab: done: 0.016834 sec","job_status":"done","queue_duration_s":2.433427,"scheduling_latency_s":2.433445,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016834,"completed_at":"2026-08-28T11:39:11.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 250.229063] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1d4ea008a8004431b54c79c","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1d4ea008a8004431b54c79c: start","job_status":"start","queue_duration_s":2.451858,"scheduling_latency_s":2.451878}1384gitlab # [ 250.241633] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:39:11.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1d4ea008a8004431b54c79c","created_at":"2026-08-28T11:39:08.377Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-44012d7a6d58477a","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c1affcc204d554d883bd9aacfbf55912","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:39:08.772Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1d4ea008a8004431b54c79c: done: 0.017587 sec","job_status":"done","queue_duration_s":2.451858,"scheduling_latency_s":2.451878,"redis_calls":13,"redis_duration_s":0.001644,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017587,"completed_at":"2026-08-28T11:39:11.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 286.810606] setup-gitlab-start[3054]: W, [2026-08-28T11:39:49.703106 #3054] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1386gitlab # [ 303.134757] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.026Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2cf3b83e8a4897ec42eba937","created_at":"2026-08-28T11:40:06.013Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70d89c9577d44604256863cbf6d278bd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-28T11:40:06.023Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2cf3b83e8a4897ec42eba937: start","job_status":"start","queue_duration_s":0.003403,"scheduling_latency_s":0.003466}1387gitlab # [ 303.292572] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2cf3b83e8a4897ec42eba937","created_at":"2026-08-28T11:40:06.013Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70d89c9577d44604256863cbf6d278bd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-28T11:40:06.023Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2cf3b83e8a4897ec42eba937: done: 0.158577 sec","job_status":"done","queue_duration_s":0.003403,"scheduling_latency_s":0.003466,"redis_calls":14,"redis_duration_s":0.058829,"redis_read_bytes":901,"redis_write_bytes":1211,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.040052,"redis_feature_flag_read_bytes":896,"redis_feature_flag_write_bytes":470,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004654,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.158577,"completed_at":"2026-08-28T11:40:06.184Z","load_balancing_strategy":"primary","db_duration_s":0.008436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 303.383438] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2acbf3e0ddfeea751d7550fb","created_at":"2026-08-28T11:40:06.040Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.064Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2acbf3e0ddfeea751d7550fb: start","job_status":"start","queue_duration_s":0.144438,"scheduling_latency_s":0.144489}1389gitlab # [ 303.478400] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2acbf3e0ddfeea751d7550fb","created_at":"2026-08-28T11:40:06.040Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.064Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2acbf3e0ddfeea751d7550fb: done: 0.16212 sec","job_status":"done","queue_duration_s":0.144438,"scheduling_latency_s":0.144489,"redis_calls":15,"redis_duration_s":0.048763999999999995,"redis_read_bytes":144,"redis_write_bytes":4075,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.009698,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_calls":2,"redis_queues_duration_s":0.001109,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.033329,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.16212,"completed_at":"2026-08-28T11:40:06.370Z","load_balancing_strategy":"primary","db_duration_s":0.008853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 303.558416] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0575d36165dbc2be6a6ffa41","created_at":"2026-08-28T11:40:06.094Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f33d0c9d7a3778540bc6140c302ac48a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.124Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0575d36165dbc2be6a6ffa41: start","job_status":"start","queue_duration_s":0.262718,"scheduling_latency_s":0.262777}1391gitlab # [ 303.575179] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.423Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1392gitlab # [ 303.579321] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"0575d36165dbc2be6a6ffa41","created_at":"2026-08-28T11:40:06.094Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f33d0c9d7a3778540bc6140c302ac48a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.124Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-0575d36165dbc2be6a6ffa41: done: 0.063565 sec","job_status":"done","queue_duration_s":0.262718,"scheduling_latency_s":0.262777,"redis_calls":7,"redis_duration_s":0.013614999999999999,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004715,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063565,"completed_at":"2026-08-28T11:40:06.450Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 303.620915] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:06.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4086444714381196ecd5279b","created_at":"2026-08-28T11:40:06.149Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.159Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4086444714381196ecd5279b: start","job_status":"start","queue_duration_s":0.338766,"scheduling_latency_s":0.338833}1394gitlab # [ 304.238252] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4086444714381196ecd5279b","created_at":"2026-08-28T11:40:06.149Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.159Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4086444714381196ecd5279b: done: 0.631319 sec","job_status":"done","queue_duration_s":0.338766,"scheduling_latency_s":0.338833,"redis_calls":60,"redis_duration_s":0.17597900000000002,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.002377,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.168724,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004878,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.275521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.631319,"completed_at":"2026-08-28T11:40:07.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 304.243811] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2b3065b65a8e60f00549ac51","created_at":"2026-08-28T11:40:06.175Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2c1c0db21e1ff510271a55acd9cc6b5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.181Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2b3065b65a8e60f00549ac51: start","job_status":"start","queue_duration_s":0.953986,"scheduling_latency_s":0.954045}1396gitlab # [ 304.354292] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2b3065b65a8e60f00549ac51","created_at":"2026-08-28T11:40:06.175Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2c1c0db21e1ff510271a55acd9cc6b5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.181Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2b3065b65a8e60f00549ac51: done: 0.109611 sec","job_status":"done","queue_duration_s":0.953986,"scheduling_latency_s":0.954045,"redis_calls":9,"redis_duration_s":0.003236,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001642,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00127954,"exclusive_lock_hold_duration_s":0.05966012,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.109611,"completed_at":"2026-08-28T11:40:07.245Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.055808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 304.409088] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.252Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"432e88877fd2e51ddda5496b","created_at":"2026-08-28T11:40:06.195Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4420f41bc20ec0262780a6f399d1da78","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.216Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-432e88877fd2e51ddda5496b: start","job_status":"start","queue_duration_s":1.035562,"scheduling_latency_s":1.035584}1398gitlab # [ 304.423599] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"432e88877fd2e51ddda5496b","created_at":"2026-08-28T11:40:06.195Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4420f41bc20ec0262780a6f399d1da78","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.216Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-432e88877fd2e51ddda5496b: done: 0.037142 sec","job_status":"done","queue_duration_s":1.035562,"scheduling_latency_s":1.035584,"redis_calls":12,"redis_duration_s":0.001769,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000431,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026782,"duration_s":0.037142,"completed_at":"2026-08-28T11:40:07.289Z","load_balancing_strategy":"primary","db_duration_s":0.009503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 304.467829] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.290Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8338d743db00dd2f82415c8d","created_at":"2026-08-28T11:40:06.253Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4bed6662aa07ad16eea6c3269126ec9f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.259Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8338d743db00dd2f82415c8d: start","job_status":"start","queue_duration_s":1.031093,"scheduling_latency_s":1.031115}1400gitlab # [ 304.495687] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8338d743db00dd2f82415c8d","created_at":"2026-08-28T11:40:06.253Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4bed6662aa07ad16eea6c3269126ec9f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.259Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8338d743db00dd2f82415c8d: done: 0.098171 sec","job_status":"done","queue_duration_s":1.031093,"scheduling_latency_s":1.031115,"redis_calls":7,"redis_duration_s":0.0010509999999999999,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.065,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.098171,"completed_at":"2026-08-28T11:40:07.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 304.537869] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4c939fe352c861c7735ab1bd","created_at":"2026-08-28T11:40:06.286Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe9e031cb0f6bd9b211d6d7145716e86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.297Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4c939fe352c861c7735ab1bd: start","job_status":"start","queue_duration_s":1.09292,"scheduling_latency_s":1.092944}1402gitlab # [ 304.824669] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4c939fe352c861c7735ab1bd","created_at":"2026-08-28T11:40:06.286Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe9e031cb0f6bd9b211d6d7145716e86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.297Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4c939fe352c861c7735ab1bd: done: 0.327169 sec","job_status":"done","queue_duration_s":1.09292,"scheduling_latency_s":1.092944,"redis_calls":11,"redis_duration_s":0.012223000000000001,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005864,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.200002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005011,"exclusive_lock_hold_duration_s":0.28744156,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.327169,"completed_at":"2026-08-28T11:40:07.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.112834,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 304.924604] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.721Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f37f52c990a4c7d52e0d3a32","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f37f52c990a4c7d52e0d3a32: start","job_status":"start","queue_duration_s":1.413144,"scheduling_latency_s":1.413174}1404gitlab # [ 304.940210] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.778Z","message":"No existing merge request to be cleaned up."}1405gitlab # [ 305.029651] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.922Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f37f52c990a4c7d52e0d3a32","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f37f52c990a4c7d52e0d3a32: done: 0.200934 sec","job_status":"done","queue_duration_s":1.413144,"scheduling_latency_s":1.413174,"redis_calls":10,"redis_duration_s":0.008629,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003717,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004912,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.115,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.115,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.200934,"completed_at":"2026-08-28T11:40:07.922Z","load_balancing_strategy":"primary","db_duration_s":0.120849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 305.120916] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:07.927Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c5987b91f19bec69229cff6a","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c5987b91f19bec69229cff6a: start","job_status":"start","queue_duration_s":1.619203,"scheduling_latency_s":1.619247}1407gitlab # [ 305.182855] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.076Z","message":"No existing merge request to be cleaned up."}1408gitlab # [ 305.236822] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.129Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c5987b91f19bec69229cff6a","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c5987b91f19bec69229cff6a: done: 0.201707 sec","job_status":"done","queue_duration_s":1.619203,"scheduling_latency_s":1.619247,"redis_calls":11,"redis_duration_s":0.010949,"redis_read_bytes":136,"redis_write_bytes":1131,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001587,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":109,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003118,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006244,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.101,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.101,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.201707,"completed_at":"2026-08-28T11:40:08.129Z","load_balancing_strategy":"primary","db_duration_s":0.197548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 305.380432] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.272Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"718d06a6221d161ad320d854","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-718d06a6221d161ad320d854: start","job_status":"start","queue_duration_s":1.963688,"scheduling_latency_s":1.963748}1410gitlab # [ 305.500120] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.393Z","message":"No existing merge request to be cleaned up."}1411gitlab # [ 305.561738] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.454Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"718d06a6221d161ad320d854","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-718d06a6221d161ad320d854: done: 0.182186 sec","job_status":"done","queue_duration_s":1.963688,"scheduling_latency_s":1.963748,"redis_calls":10,"redis_duration_s":0.013415,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006273,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.007142,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.182186,"completed_at":"2026-08-28T11:40:08.454Z","load_balancing_strategy":"primary","db_duration_s":0.006856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 305.709854] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.602Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dd5ff5d0af8c26ca314fffb3","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dd5ff5d0af8c26ca314fffb3: start","job_status":"start","queue_duration_s":2.294283,"scheduling_latency_s":2.294348}1413gitlab # [ 305.788732] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.682Z","message":"No existing merge request to be cleaned up."}1414gitlab # [ 305.834863] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.727Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dd5ff5d0af8c26ca314fffb3","created_at":"2026-08-28T11:40:06.264Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-57cb7a55aba743fb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"794ba40189c39e2b191c91a116e42522","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dd5ff5d0af8c26ca314fffb3: done: 0.124653 sec","job_status":"done","queue_duration_s":2.294283,"scheduling_latency_s":2.294348,"redis_calls":10,"redis_duration_s":0.017879,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008371,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.009508,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.124653,"completed_at":"2026-08-28T11:40:08.727Z","load_balancing_strategy":"primary","db_duration_s":0.015745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 305.927334] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:08.820Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f181df4d990408cf24f4d020","created_at":"2026-08-28T11:40:06.328Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1f3291d7ec5c16d32681c8ccf93dd10","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.335Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f181df4d990408cf24f4d020: start","job_status":"start","queue_duration_s":2.484488,"scheduling_latency_s":2.484546}1416gitlab # [ 306.141440] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"f181df4d990408cf24f4d020","created_at":"2026-08-28T11:40:06.328Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1f3291d7ec5c16d32681c8ccf93dd10","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.335Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-f181df4d990408cf24f4d020: done: 0.212661 sec","job_status":"done","queue_duration_s":2.484488,"scheduling_latency_s":2.484546,"redis_calls":10,"redis_duration_s":0.016937999999999998,"redis_read_bytes":135,"redis_write_bytes":1302,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001768,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":133,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007009,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00299302,"exclusive_lock_hold_duration_s":0.04131416,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.212661,"completed_at":"2026-08-28T11:40:09.032Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 306.242090] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"947f0b708c3899bb35490cf2","created_at":"2026-08-28T11:40:06.373Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c637f83660b377afde791fb5fd396fc","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.395Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-947f0b708c3899bb35490cf2: start","job_status":"start","queue_duration_s":2.739909,"scheduling_latency_s":2.73997}1418gitlab # [ 306.395001] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"947f0b708c3899bb35490cf2","created_at":"2026-08-28T11:40:06.373Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c637f83660b377afde791fb5fd396fc","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.395Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-947f0b708c3899bb35490cf2: done: 0.151426 sec","job_status":"done","queue_duration_s":2.739909,"scheduling_latency_s":2.73997,"redis_calls":10,"redis_duration_s":0.015983999999999998,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003411,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009946,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002627,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0014569,"duration_s":0.151426,"completed_at":"2026-08-28T11:40:09.286Z","load_balancing_strategy":"primary","db_duration_s":0.012343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 306.491289] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"53ffe78e1f8a101bfda0792f","created_at":"2026-08-28T11:40:06.424Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f07e8d8cd82a42d2b1587ad49f0b48b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.437Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-53ffe78e1f8a101bfda0792f: start","job_status":"start","queue_duration_s":2.946921,"scheduling_latency_s":2.946977}1420gitlab # [ 306.570433] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.463Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1421gitlab # [ 306.610598] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"53ffe78e1f8a101bfda0792f","created_at":"2026-08-28T11:40:06.424Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f07e8d8cd82a42d2b1587ad49f0b48b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.437Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-53ffe78e1f8a101bfda0792f: done: 0.118908 sec","job_status":"done","queue_duration_s":2.946921,"scheduling_latency_s":2.946977,"redis_calls":9,"redis_duration_s":0.011464,"redis_read_bytes":261,"redis_write_bytes":1075,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002988,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":249,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.118908,"completed_at":"2026-08-28T11:40:09.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 306.702183] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"1f462fa8e8bb1b407318c52e","created_at":"2026-08-28T11:40:06.491Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e4b9957d3e1a5c413e7630ccf174d53","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.507Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-1f462fa8e8bb1b407318c52e: start","job_status":"start","queue_duration_s":3.078329,"scheduling_latency_s":3.078386}1423gitlab # [ 306.826585] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"1f462fa8e8bb1b407318c52e","created_at":"2026-08-28T11:40:06.491Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e4b9957d3e1a5c413e7630ccf174d53","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.507Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-1f462fa8e8bb1b407318c52e: done: 0.133428 sec","job_status":"done","queue_duration_s":3.078329,"scheduling_latency_s":3.078386,"redis_calls":20,"redis_duration_s":0.007166,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002255,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.001279,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002976,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005791,"duration_s":0.133428,"completed_at":"2026-08-28T11:40:09.719Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 306.929689] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"218ec27b374f295267670f39","created_at":"2026-08-28T11:40:06.542Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c97f9ea51a015315aeff8e6771c5557","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.563Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-218ec27b374f295267670f39: start","job_status":"start","queue_duration_s":3.158849,"scheduling_latency_s":3.158888}1425gitlab # [ 306.951051] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.768Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"218ec27b374f295267670f39","created_at":"2026-08-28T11:40:06.542Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c97f9ea51a015315aeff8e6771c5557","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.563Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-218ec27b374f295267670f39: done: 0.046407 sec","job_status":"done","queue_duration_s":3.158849,"scheduling_latency_s":3.158888,"redis_calls":9,"redis_duration_s":0.003203,"redis_read_bytes":261,"redis_write_bytes":1073,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000576,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":247,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.046407,"completed_at":"2026-08-28T11:40:09.768Z","load_balancing_strategy":"primary","db_duration_s":0.016382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 306.999079] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"3ddbb399926281c1acb811ea","created_at":"2026-08-28T11:40:06.600Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efd125c17ebf03602da7c2f1a730d553","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.641Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-3ddbb399926281c1acb811ea: start","job_status":"start","queue_duration_s":3.130094,"scheduling_latency_s":3.130128}1427gitlab # [ 307.013047] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"3ddbb399926281c1acb811ea","created_at":"2026-08-28T11:40:06.600Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efd125c17ebf03602da7c2f1a730d553","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.641Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-3ddbb399926281c1acb811ea: done: 0.053976 sec","job_status":"done","queue_duration_s":3.130094,"scheduling_latency_s":3.130128,"redis_calls":10,"redis_duration_s":0.0029360000000000002,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001688,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000725,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053864,"duration_s":0.053976,"completed_at":"2026-08-28T11:40:09.825Z","load_balancing_strategy":"primary","db_duration_s":0.007769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 307.054987] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.827Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"84f4311a609a018d9dcea50c","created_at":"2026-08-28T11:40:06.673Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58bb26c02eec578e4121ddab9b0e26b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.684Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-84f4311a609a018d9dcea50c: start","job_status":"start","queue_duration_s":3.142564,"scheduling_latency_s":3.14259}1429gitlab # [ 307.069708] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.840Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1430gitlab # [ 307.072891] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"84f4311a609a018d9dcea50c","created_at":"2026-08-28T11:40:06.673Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58bb26c02eec578e4121ddab9b0e26b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.684Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-84f4311a609a018d9dcea50c: done: 0.020648 sec","job_status":"done","queue_duration_s":3.142564,"scheduling_latency_s":3.14259,"redis_calls":9,"redis_duration_s":0.001755,"redis_read_bytes":261,"redis_write_bytes":1002,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000379,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":231,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020648,"completed_at":"2026-08-28T11:40:09.848Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 307.116188] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.849Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"5a2b49d4effb83899d18ae71","created_at":"2026-08-28T11:40:06.720Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"63204c8b38319439edfcceaaa3199426","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.755Z","job_size_bytes":3,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-5a2b49d4effb83899d18ae71: start","job_status":"start","queue_duration_s":3.093847,"scheduling_latency_s":3.093869}1432gitlab # [ 307.130113] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.871Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"5a2b49d4effb83899d18ae71","created_at":"2026-08-28T11:40:06.720Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"63204c8b38319439edfcceaaa3199426","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.755Z","job_size_bytes":3,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-5a2b49d4effb83899d18ae71: done: 0.021623 sec","job_status":"done","queue_duration_s":3.093847,"scheduling_latency_s":3.093869,"redis_calls":10,"redis_duration_s":0.0016749999999999998,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000346,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000343,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034066,"duration_s":0.021623,"completed_at":"2026-08-28T11:40:09.871Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 307.171840] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"beab5420f2712a76bbe63fc1","created_at":"2026-08-28T11:40:06.784Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d38055bf863ee8289a11350aa13564d","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.810Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-beab5420f2712a76bbe63fc1: start","job_status":"start","queue_duration_s":3.061743,"scheduling_latency_s":3.061763}1434gitlab # [ 307.186733] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"beab5420f2712a76bbe63fc1","created_at":"2026-08-28T11:40:06.784Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d38055bf863ee8289a11350aa13564d","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.810Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-beab5420f2712a76bbe63fc1: done: 0.042131 sec","job_status":"done","queue_duration_s":3.061743,"scheduling_latency_s":3.061763,"redis_calls":13,"redis_duration_s":0.004685,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000339,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001051,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.003295,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044646,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.042131,"completed_at":"2026-08-28T11:40:09.914Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 307.231562] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"538b6eb2b98625b1bb8b5e14","created_at":"2026-08-28T11:40:06.840Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4767672ed9a958c71b4ea895b8ce28a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.851Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-538b6eb2b98625b1bb8b5e14: start","job_status":"start","queue_duration_s":3.064828,"scheduling_latency_s":3.064848}1436gitlab # [ 307.246415] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"538b6eb2b98625b1bb8b5e14","created_at":"2026-08-28T11:40:06.840Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4767672ed9a958c71b4ea895b8ce28a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.851Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-538b6eb2b98625b1bb8b5e14: done: 0.058095 sec","job_status":"done","queue_duration_s":3.064828,"scheduling_latency_s":3.064848,"redis_calls":12,"redis_duration_s":0.001799,"redis_read_bytes":391,"redis_write_bytes":1466,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000411,"redis_feature_flag_read_bytes":384,"redis_feature_flag_write_bytes":304,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050904,"exclusive_lock_hold_duration_s":0.04145426,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.058095,"completed_at":"2026-08-28T11:40:09.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 307.291216] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a0a8ff3a56bbbdb43f01b224","created_at":"2026-08-28T11:40:06.914Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20dfc680c5ab1c1a4ba55f195c4887a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.924Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a0a8ff3a56bbbdb43f01b224: start","job_status":"start","queue_duration_s":3.051176,"scheduling_latency_s":3.051193}1438gitlab # [ 307.307420] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.989Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a0a8ff3a56bbbdb43f01b224","created_at":"2026-08-28T11:40:06.914Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20dfc680c5ab1c1a4ba55f195c4887a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.924Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a0a8ff3a56bbbdb43f01b224: done: 0.014194 sec","job_status":"done","queue_duration_s":3.051176,"scheduling_latency_s":3.051193,"redis_calls":10,"redis_duration_s":0.001294,"redis_read_bytes":389,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000367,"redis_feature_flag_read_bytes":384,"redis_feature_flag_write_bytes":316,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014194,"completed_at":"2026-08-28T11:40:09.989Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 307.350012] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:09.991Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fd98857a4552ff0159b2d872","created_at":"2026-08-28T11:40:06.950Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"871e6aec2e936e7a363dad46aa15fb92","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.960Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fd98857a4552ff0159b2d872: start","job_status":"start","queue_duration_s":3.030297,"scheduling_latency_s":3.030314}1440gitlab # [ 307.363723] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fd98857a4552ff0159b2d872","created_at":"2026-08-28T11:40:06.950Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"871e6aec2e936e7a363dad46aa15fb92","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:06.960Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fd98857a4552ff0159b2d872: done: 0.017993 sec","job_status":"done","queue_duration_s":3.030297,"scheduling_latency_s":3.030314,"redis_calls":11,"redis_duration_s":0.001518,"redis_read_bytes":263,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":217,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00046996,"exclusive_lock_hold_duration_s":0.00402872,"duration_s":0.017993,"completed_at":"2026-08-28T11:40:10.008Z","load_balancing_strategy":"primary","db_duration_s":0.00182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 307.406401] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.010Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"a071b4c2cd92a3b7dfc4bb6d","created_at":"2026-08-28T11:40:07.006Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21b7b2dd8f0512fa34c975c0519c8bdb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.028Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-a071b4c2cd92a3b7dfc4bb6d: start","job_status":"start","queue_duration_s":2.981955,"scheduling_latency_s":2.981973}1442gitlab # [ 307.420786] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"a071b4c2cd92a3b7dfc4bb6d","created_at":"2026-08-28T11:40:07.006Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21b7b2dd8f0512fa34c975c0519c8bdb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.028Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-a071b4c2cd92a3b7dfc4bb6d: done: 0.022321 sec","job_status":"done","queue_duration_s":2.981955,"scheduling_latency_s":2.981973,"redis_calls":14,"redis_duration_s":0.001874,"redis_read_bytes":393,"redis_write_bytes":1385,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":386,"redis_feature_flag_write_bytes":528,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000205,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002298,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.022321,"completed_at":"2026-08-28T11:40:10.032Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 307.464880] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4973bfaac6a6fe2305d98b8b","created_at":"2026-08-28T11:40:07.066Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dfa8cd73b0dd7b5040c6de6e7047dde1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.077Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4973bfaac6a6fe2305d98b8b: start","job_status":"start","queue_duration_s":2.956697,"scheduling_latency_s":2.956713}1444gitlab # [ 307.478836] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4973bfaac6a6fe2305d98b8b","created_at":"2026-08-28T11:40:07.066Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dfa8cd73b0dd7b5040c6de6e7047dde1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.077Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4973bfaac6a6fe2305d98b8b: done: 0.016392 sec","job_status":"done","queue_duration_s":2.956697,"scheduling_latency_s":2.956713,"redis_calls":9,"redis_duration_s":0.001422,"redis_read_bytes":261,"redis_write_bytes":874,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":193,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016392,"completed_at":"2026-08-28T11:40:10.050Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 307.521577] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3836f270b06ebac402c8238","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3836f270b06ebac402c8238: start","job_status":"start","queue_duration_s":2.953041,"scheduling_latency_s":2.953058}1446gitlab # [ 307.533934] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3836f270b06ebac402c8238","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3836f270b06ebac402c8238: done: 0.017129 sec","job_status":"done","queue_duration_s":2.953041,"scheduling_latency_s":2.953058,"redis_calls":17,"redis_duration_s":0.001976,"redis_read_bytes":522,"redis_write_bytes":1453,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":296,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017129,"completed_at":"2026-08-28T11:40:10.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 307.575002] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a40bc69ad263ad8016e6e0fa","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a40bc69ad263ad8016e6e0fa: start","job_status":"start","queue_duration_s":2.971224,"scheduling_latency_s":2.97124}1448gitlab # [ 307.587686] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a40bc69ad263ad8016e6e0fa","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a40bc69ad263ad8016e6e0fa: done: 0.014198 sec","job_status":"done","queue_duration_s":2.971224,"scheduling_latency_s":2.97124,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014198,"completed_at":"2026-08-28T11:40:10.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 307.627469] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1606a01090182d60e32804d9","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1606a01090182d60e32804d9: start","job_status":"start","queue_duration_s":2.986671,"scheduling_latency_s":2.986687}1450gitlab # [ 307.640099] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1606a01090182d60e32804d9","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1606a01090182d60e32804d9: done: 0.014286 sec","job_status":"done","queue_duration_s":2.986671,"scheduling_latency_s":2.986687,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014286,"completed_at":"2026-08-28T11:40:10.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 307.679188] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e816e6a4573cf312cb0e786","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e816e6a4573cf312cb0e786: start","job_status":"start","queue_duration_s":3.002296,"scheduling_latency_s":3.002314}1452gitlab # [ 307.691796] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e816e6a4573cf312cb0e786","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e816e6a4573cf312cb0e786: done: 0.014382 sec","job_status":"done","queue_duration_s":3.002296,"scheduling_latency_s":3.002314,"redis_calls":13,"redis_duration_s":0.001439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014382,"completed_at":"2026-08-28T11:40:10.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 307.732710] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"875be4ab39b6c47c855869a3","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-875be4ab39b6c47c855869a3: start","job_status":"start","queue_duration_s":3.017902,"scheduling_latency_s":3.017918}1454gitlab # [ 307.744902] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"875be4ab39b6c47c855869a3","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-875be4ab39b6c47c855869a3: done: 0.013967 sec","job_status":"done","queue_duration_s":3.017902,"scheduling_latency_s":3.017918,"redis_calls":13,"redis_duration_s":0.0014160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013967,"completed_at":"2026-08-28T11:40:10.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 307.784537] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c680922a5f7ba77541c25cab","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c680922a5f7ba77541c25cab: start","job_status":"start","queue_duration_s":3.033098,"scheduling_latency_s":3.033115}1456gitlab # [ 307.796889] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c680922a5f7ba77541c25cab","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c680922a5f7ba77541c25cab: done: 0.01439 sec","job_status":"done","queue_duration_s":3.033098,"scheduling_latency_s":3.033115,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01439,"completed_at":"2026-08-28T11:40:10.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 307.836233] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a4981700ef01c4c5fe5c7fd","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a4981700ef01c4c5fe5c7fd: start","job_status":"start","queue_duration_s":3.048855,"scheduling_latency_s":3.048872}1458gitlab # [ 307.848658] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a4981700ef01c4c5fe5c7fd","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a4981700ef01c4c5fe5c7fd: done: 0.014063 sec","job_status":"done","queue_duration_s":3.048855,"scheduling_latency_s":3.048872,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014063,"completed_at":"2026-08-28T11:40:10.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 307.888123] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f2e671d5d4785775d1ddf2d","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f2e671d5d4785775d1ddf2d: start","job_status":"start","queue_duration_s":3.064003,"scheduling_latency_s":3.06402}1460gitlab # [ 307.900417] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f2e671d5d4785775d1ddf2d","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f2e671d5d4785775d1ddf2d: done: 0.014596 sec","job_status":"done","queue_duration_s":3.064003,"scheduling_latency_s":3.06402,"redis_calls":13,"redis_duration_s":0.0014190000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014596,"completed_at":"2026-08-28T11:40:10.176Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 307.941026] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e846c369106faeb6dbe4d6","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e846c369106faeb6dbe4d6: start","job_status":"start","queue_duration_s":3.079687,"scheduling_latency_s":3.079704}1462gitlab # [ 307.953381] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e846c369106faeb6dbe4d6","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e846c369106faeb6dbe4d6: done: 0.014081 sec","job_status":"done","queue_duration_s":3.079687,"scheduling_latency_s":3.079704,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014081,"completed_at":"2026-08-28T11:40:10.192Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 307.992373] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c98ec42bf4fc881f32cbfa6f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c98ec42bf4fc881f32cbfa6f: start","job_status":"start","queue_duration_s":3.094824,"scheduling_latency_s":3.094841}1464gitlab # [ 308.004647] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c98ec42bf4fc881f32cbfa6f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c98ec42bf4fc881f32cbfa6f: done: 0.014511 sec","job_status":"done","queue_duration_s":3.094824,"scheduling_latency_s":3.094841,"redis_calls":13,"redis_duration_s":0.0015079999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014511,"completed_at":"2026-08-28T11:40:10.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 308.044018] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f55332939511b3d45da95a6e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f55332939511b3d45da95a6e: start","job_status":"start","queue_duration_s":3.110444,"scheduling_latency_s":3.110462}1466gitlab # [ 308.056248] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f55332939511b3d45da95a6e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f55332939511b3d45da95a6e: done: 0.014092 sec","job_status":"done","queue_duration_s":3.110444,"scheduling_latency_s":3.110462,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014092,"completed_at":"2026-08-28T11:40:10.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 308.095047] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"456c14ecd5b642079358997b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-456c14ecd5b642079358997b: start","job_status":"start","queue_duration_s":3.125578,"scheduling_latency_s":3.125593}1468gitlab # [ 308.108177] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"456c14ecd5b642079358997b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-456c14ecd5b642079358997b: done: 0.013842 sec","job_status":"done","queue_duration_s":3.125578,"scheduling_latency_s":3.125593,"redis_calls":13,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013842,"completed_at":"2026-08-28T11:40:10.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 308.148934] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6da7dc5ceaa501f7a35bdea","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6da7dc5ceaa501f7a35bdea: start","job_status":"start","queue_duration_s":3.140689,"scheduling_latency_s":3.140706}1470gitlab # [ 308.161110] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6da7dc5ceaa501f7a35bdea","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6da7dc5ceaa501f7a35bdea: done: 0.014485 sec","job_status":"done","queue_duration_s":3.140689,"scheduling_latency_s":3.140706,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014485,"completed_at":"2026-08-28T11:40:10.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 308.200356] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0afee826ee4f907794a2424","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0afee826ee4f907794a2424: start","job_status":"start","queue_duration_s":3.156421,"scheduling_latency_s":3.156438}1472gitlab # [ 308.212709] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0afee826ee4f907794a2424","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0afee826ee4f907794a2424: done: 0.01395 sec","job_status":"done","queue_duration_s":3.156421,"scheduling_latency_s":3.156438,"redis_calls":13,"redis_duration_s":0.0014260000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01395,"completed_at":"2026-08-28T11:40:10.268Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 308.252199] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fac32e96c3e4ebad7a008d89","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fac32e96c3e4ebad7a008d89: start","job_status":"start","queue_duration_s":3.171604,"scheduling_latency_s":3.171619}1474gitlab # [ 308.264659] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fac32e96c3e4ebad7a008d89","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fac32e96c3e4ebad7a008d89: done: 0.01408 sec","job_status":"done","queue_duration_s":3.171604,"scheduling_latency_s":3.171619,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01408,"completed_at":"2026-08-28T11:40:10.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 308.306454] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be541bb312e050c329aa3fce","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be541bb312e050c329aa3fce: start","job_status":"start","queue_duration_s":3.186911,"scheduling_latency_s":3.186928}1476gitlab # [ 308.320178] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be541bb312e050c329aa3fce","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be541bb312e050c329aa3fce: done: 0.015191 sec","job_status":"done","queue_duration_s":3.186911,"scheduling_latency_s":3.186928,"redis_calls":13,"redis_duration_s":0.001726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015191,"completed_at":"2026-08-28T11:40:10.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 308.359562] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3df2b6f45cd108ced824460","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3df2b6f45cd108ced824460: start","job_status":"start","queue_duration_s":3.20345,"scheduling_latency_s":3.203468}1478gitlab # [ 308.372104] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3df2b6f45cd108ced824460","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3df2b6f45cd108ced824460: done: 0.014545 sec","job_status":"done","queue_duration_s":3.20345,"scheduling_latency_s":3.203468,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014545,"completed_at":"2026-08-28T11:40:10.316Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 308.411823] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7117b58ad1cc3a2716e9b82c","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7117b58ad1cc3a2716e9b82c: start","job_status":"start","queue_duration_s":3.219089,"scheduling_latency_s":3.219105}1480gitlab # [ 308.424599] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7117b58ad1cc3a2716e9b82c","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7117b58ad1cc3a2716e9b82c: done: 0.014853 sec","job_status":"done","queue_duration_s":3.219089,"scheduling_latency_s":3.219105,"redis_calls":13,"redis_duration_s":0.0015049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014853,"completed_at":"2026-08-28T11:40:10.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 308.463835] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2abf0ccbd7a9af3520a8b96e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2abf0ccbd7a9af3520a8b96e: start","job_status":"start","queue_duration_s":3.235073,"scheduling_latency_s":3.23509}1482gitlab # [ 308.476612] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2abf0ccbd7a9af3520a8b96e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2abf0ccbd7a9af3520a8b96e: done: 0.016256 sec","job_status":"done","queue_duration_s":3.235073,"scheduling_latency_s":3.23509,"redis_calls":13,"redis_duration_s":0.001854,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016256,"completed_at":"2026-08-28T11:40:10.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 308.516043] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffc8efb43b6a81bc0e463d77","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffc8efb43b6a81bc0e463d77: start","job_status":"start","queue_duration_s":3.252574,"scheduling_latency_s":3.252591}1484gitlab # [ 308.529617] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffc8efb43b6a81bc0e463d77","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffc8efb43b6a81bc0e463d77: done: 0.014474 sec","job_status":"done","queue_duration_s":3.252574,"scheduling_latency_s":3.252591,"redis_calls":13,"redis_duration_s":0.001451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014474,"completed_at":"2026-08-28T11:40:10.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 308.568623] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2ac7f7723a1957bc51f879f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2ac7f7723a1957bc51f879f: start","job_status":"start","queue_duration_s":3.268246,"scheduling_latency_s":3.268266}1486gitlab # [ 308.580974] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2ac7f7723a1957bc51f879f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2ac7f7723a1957bc51f879f: done: 0.015442 sec","job_status":"done","queue_duration_s":3.268246,"scheduling_latency_s":3.268266,"redis_calls":14,"redis_duration_s":0.001629,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000123,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015442,"completed_at":"2026-08-28T11:40:10.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 308.622115] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eae60a39638a732c4538f5d3","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eae60a39638a732c4538f5d3: start","job_status":"start","queue_duration_s":3.284809,"scheduling_latency_s":3.284828}1488gitlab # [ 308.634471] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eae60a39638a732c4538f5d3","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eae60a39638a732c4538f5d3: done: 0.014645 sec","job_status":"done","queue_duration_s":3.284809,"scheduling_latency_s":3.284828,"redis_calls":13,"redis_duration_s":0.0014880000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014645,"completed_at":"2026-08-28T11:40:10.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 308.673969] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00ebe14e04bc3fbc672ddb2f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00ebe14e04bc3fbc672ddb2f: start","job_status":"start","queue_duration_s":3.300642,"scheduling_latency_s":3.300664}1490gitlab # [ 308.686472] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00ebe14e04bc3fbc672ddb2f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00ebe14e04bc3fbc672ddb2f: done: 0.014927 sec","job_status":"done","queue_duration_s":3.300642,"scheduling_latency_s":3.300664,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014927,"completed_at":"2026-08-28T11:40:10.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 308.726816] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b979d71e0dfa0ffb396d8e36","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b979d71e0dfa0ffb396d8e36: start","job_status":"start","queue_duration_s":3.316799,"scheduling_latency_s":3.31682}1492gitlab # [ 308.739356] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b979d71e0dfa0ffb396d8e36","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b979d71e0dfa0ffb396d8e36: done: 0.014705 sec","job_status":"done","queue_duration_s":3.316799,"scheduling_latency_s":3.31682,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014705,"completed_at":"2026-08-28T11:40:10.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 308.778886] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd99d26155605432c8595160","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd99d26155605432c8595160: start","job_status":"start","queue_duration_s":3.332917,"scheduling_latency_s":3.332938}1494gitlab # [ 308.791469] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd99d26155605432c8595160","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd99d26155605432c8595160: done: 0.014782 sec","job_status":"done","queue_duration_s":3.332917,"scheduling_latency_s":3.332938,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014782,"completed_at":"2026-08-28T11:40:10.445Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 308.830761] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3134dd2425594f301a65119","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3134dd2425594f301a65119: start","job_status":"start","queue_duration_s":3.34929,"scheduling_latency_s":3.349312}1496gitlab # [ 308.843291] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3134dd2425594f301a65119","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3134dd2425594f301a65119: done: 0.014927 sec","job_status":"done","queue_duration_s":3.34929,"scheduling_latency_s":3.349312,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014927,"completed_at":"2026-08-28T11:40:10.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 308.882818] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65382a684d4af21eac26607e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65382a684d4af21eac26607e: start","job_status":"start","queue_duration_s":3.365507,"scheduling_latency_s":3.365528}1498gitlab # [ 308.895379] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65382a684d4af21eac26607e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65382a684d4af21eac26607e: done: 0.014359 sec","job_status":"done","queue_duration_s":3.365507,"scheduling_latency_s":3.365528,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000683,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014359,"completed_at":"2026-08-28T11:40:10.478Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 308.935915] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fa69d18bc78a61604d921c0","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fa69d18bc78a61604d921c0: start","job_status":"start","queue_duration_s":3.381198,"scheduling_latency_s":3.381219}1500gitlab # [ 308.948633] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fa69d18bc78a61604d921c0","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fa69d18bc78a61604d921c0: done: 0.014452 sec","job_status":"done","queue_duration_s":3.381198,"scheduling_latency_s":3.381219,"redis_calls":13,"redis_duration_s":0.001619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014452,"completed_at":"2026-08-28T11:40:10.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 308.988074] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65edd01739a7c7edae38ad5b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65edd01739a7c7edae38ad5b: start","job_status":"start","queue_duration_s":3.397046,"scheduling_latency_s":3.397067}1502gitlab # [ 309.000379] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65edd01739a7c7edae38ad5b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65edd01739a7c7edae38ad5b: done: 0.014484 sec","job_status":"done","queue_duration_s":3.397046,"scheduling_latency_s":3.397067,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014484,"completed_at":"2026-08-28T11:40:10.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 309.039453] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58beff7e3797fe45366efa90","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58beff7e3797fe45366efa90: start","job_status":"start","queue_duration_s":3.41273,"scheduling_latency_s":3.412753}1504gitlab # [ 309.052074] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58beff7e3797fe45366efa90","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58beff7e3797fe45366efa90: done: 0.014921 sec","job_status":"done","queue_duration_s":3.41273,"scheduling_latency_s":3.412753,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014921,"completed_at":"2026-08-28T11:40:10.525Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 309.091555] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b72034b2bd6f63ef2b344e92","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b72034b2bd6f63ef2b344e92: start","job_status":"start","queue_duration_s":3.428843,"scheduling_latency_s":3.428864}1506gitlab # [ 309.104200] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b72034b2bd6f63ef2b344e92","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b72034b2bd6f63ef2b344e92: done: 0.015218 sec","job_status":"done","queue_duration_s":3.428843,"scheduling_latency_s":3.428864,"redis_calls":13,"redis_duration_s":0.0015149999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015218,"completed_at":"2026-08-28T11:40:10.542Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 309.145078] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5311bca83cf815f5a858dd","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5311bca83cf815f5a858dd: start","job_status":"start","queue_duration_s":3.445254,"scheduling_latency_s":3.445277}1508gitlab # [ 309.157455] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5311bca83cf815f5a858dd","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5311bca83cf815f5a858dd: done: 0.01692 sec","job_status":"done","queue_duration_s":3.445254,"scheduling_latency_s":3.445277,"redis_calls":13,"redis_duration_s":0.001567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01692,"completed_at":"2026-08-28T11:40:10.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 309.196680] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a728ce273fb48c289861f6ac","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a728ce273fb48c289861f6ac: start","job_status":"start","queue_duration_s":3.463403,"scheduling_latency_s":3.463426}1510gitlab # [ 309.208964] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a728ce273fb48c289861f6ac","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a728ce273fb48c289861f6ac: done: 0.014648 sec","job_status":"done","queue_duration_s":3.463403,"scheduling_latency_s":3.463426,"redis_calls":13,"redis_duration_s":0.0014949999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014648,"completed_at":"2026-08-28T11:40:10.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 309.248245] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcc2273b72d73a71b6bd35f6","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcc2273b72d73a71b6bd35f6: start","job_status":"start","queue_duration_s":3.479126,"scheduling_latency_s":3.479143}1512gitlab # [ 309.260783] redis[790]: 10 changes in 300 seconds. Saving...1513gitlab # [ 309.261792] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcc2273b72d73a71b6bd35f6","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcc2273b72d73a71b6bd35f6: done: 0.01459 sec","job_status":"done","queue_duration_s":3.479126,"scheduling_latency_s":3.479143,"redis_calls":13,"redis_duration_s":0.001485,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01459,"completed_at":"2026-08-28T11:40:10.592Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 309.300844] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"479a65acd75a62fc2fdc6b8b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-479a65acd75a62fc2fdc6b8b: start","job_status":"start","queue_duration_s":3.494809,"scheduling_latency_s":3.494825}1515gitlab # [ 309.313209] redis[790]: Background saving started by pid 33421516gitlab # [ 309.314160] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"479a65acd75a62fc2fdc6b8b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-479a65acd75a62fc2fdc6b8b: done: 0.01463 sec","job_status":"done","queue_duration_s":3.494809,"scheduling_latency_s":3.494825,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01463,"completed_at":"2026-08-28T11:40:10.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 309.354275] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffe96cc735ecf439e5f02602","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffe96cc735ecf439e5f02602: start","job_status":"start","queue_duration_s":3.510534,"scheduling_latency_s":3.510549}1518gitlab # [ 309.366905] redis[3342]: BGSAVE done, 463 keys saved, 0 keys skipped, 122809 bytes written.1519gitlab # [ 309.368191] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffe96cc735ecf439e5f02602","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffe96cc735ecf439e5f02602: done: 0.014513 sec","job_status":"done","queue_duration_s":3.510534,"scheduling_latency_s":3.510549,"redis_calls":13,"redis_duration_s":0.0014429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014513,"completed_at":"2026-08-28T11:40:10.623Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 309.407390] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f68b79c544281002ac022c7","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f68b79c544281002ac022c7: start","job_status":"start","queue_duration_s":3.526335,"scheduling_latency_s":3.526351}1521gitlab # [ 309.419929] redis[3342]: DB saved on disk1522gitlab # [ 309.421072] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f68b79c544281002ac022c7","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f68b79c544281002ac022c7: done: 0.014617 sec","job_status":"done","queue_duration_s":3.526335,"scheduling_latency_s":3.526351,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014617,"completed_at":"2026-08-28T11:40:10.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 309.460089] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95a454c3b92e46fca3f7820d","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95a454c3b92e46fca3f7820d: start","job_status":"start","queue_duration_s":3.542232,"scheduling_latency_s":3.54225}1524gitlab # [ 309.472129] redis[3342]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1525gitlab # [ 309.473234] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95a454c3b92e46fca3f7820d","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95a454c3b92e46fca3f7820d: done: 0.039614 sec","job_status":"done","queue_duration_s":3.542232,"scheduling_latency_s":3.54225,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039614,"completed_at":"2026-08-28T11:40:10.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 309.512299] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"451488ccbf51b865dd041605","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-451488ccbf51b865dd041605: start","job_status":"start","queue_duration_s":3.582913,"scheduling_latency_s":3.58293}1527gitlab # [ 309.524479] redis[790]: Background saving terminated with success1528gitlab # [ 309.525443] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"451488ccbf51b865dd041605","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-451488ccbf51b865dd041605: done: 0.013613 sec","job_status":"done","queue_duration_s":3.582913,"scheduling_latency_s":3.58293,"redis_calls":13,"redis_duration_s":0.001394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013613,"completed_at":"2026-08-28T11:40:10.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 309.566180] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb0e1fcc7624170cfe651e9c","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb0e1fcc7624170cfe651e9c: start","job_status":"start","queue_duration_s":3.598692,"scheduling_latency_s":3.59871}1530gitlab # [ 309.603903] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb0e1fcc7624170cfe651e9c","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb0e1fcc7624170cfe651e9c: done: 0.014999 sec","job_status":"done","queue_duration_s":3.598692,"scheduling_latency_s":3.59871,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014999,"completed_at":"2026-08-28T11:40:10.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 309.674402] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bac5df11ad89778e9e4e0a1","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bac5df11ad89778e9e4e0a1: start","job_status":"start","queue_duration_s":3.614761,"scheduling_latency_s":3.614777}1532gitlab # [ 309.688694] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bac5df11ad89778e9e4e0a1","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bac5df11ad89778e9e4e0a1: done: 0.014811 sec","job_status":"done","queue_duration_s":3.614761,"scheduling_latency_s":3.614777,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014811,"completed_at":"2026-08-28T11:40:10.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 309.728507] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58a0cf9e6748bc42a01c4ea7","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58a0cf9e6748bc42a01c4ea7: start","job_status":"start","queue_duration_s":3.630654,"scheduling_latency_s":3.63067}1534gitlab # [ 309.741000] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58a0cf9e6748bc42a01c4ea7","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58a0cf9e6748bc42a01c4ea7: done: 0.015095 sec","job_status":"done","queue_duration_s":3.630654,"scheduling_latency_s":3.63067,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015095,"completed_at":"2026-08-28T11:40:10.744Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 309.783198] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27062aee6672edef88436e19","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27062aee6672edef88436e19: start","job_status":"start","queue_duration_s":3.64871,"scheduling_latency_s":3.648728}1536gitlab # [ 309.796478] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27062aee6672edef88436e19","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27062aee6672edef88436e19: done: 0.015029 sec","job_status":"done","queue_duration_s":3.64871,"scheduling_latency_s":3.648728,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015029,"completed_at":"2026-08-28T11:40:10.762Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 309.836346] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ee1909bec4f584062c73572","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ee1909bec4f584062c73572: start","job_status":"start","queue_duration_s":3.6649,"scheduling_latency_s":3.664917}1538gitlab # [ 309.848599] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ee1909bec4f584062c73572","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ee1909bec4f584062c73572: done: 0.01487 sec","job_status":"done","queue_duration_s":3.6649,"scheduling_latency_s":3.664917,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01487,"completed_at":"2026-08-28T11:40:10.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 309.887645] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ab0ab416771aaeabb2a80f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ab0ab416771aaeabb2a80f: start","job_status":"start","queue_duration_s":3.68086,"scheduling_latency_s":3.680877}1540gitlab # [ 309.900182] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ab0ab416771aaeabb2a80f","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ab0ab416771aaeabb2a80f: done: 0.014726 sec","job_status":"done","queue_duration_s":3.68086,"scheduling_latency_s":3.680877,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014726,"completed_at":"2026-08-28T11:40:10.793Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 309.939322] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3559e5f06c35b44b47b137b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3559e5f06c35b44b47b137b: start","job_status":"start","queue_duration_s":3.696763,"scheduling_latency_s":3.69678}1542gitlab # [ 309.952711] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3559e5f06c35b44b47b137b","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3559e5f06c35b44b47b137b: done: 0.015776 sec","job_status":"done","queue_duration_s":3.696763,"scheduling_latency_s":3.69678,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015776,"completed_at":"2026-08-28T11:40:10.810Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 309.992227] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d240652dd1272d3b8c94d44e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d240652dd1272d3b8c94d44e: start","job_status":"start","queue_duration_s":3.713681,"scheduling_latency_s":3.713697}1544gitlab # [ 310.004677] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d240652dd1272d3b8c94d44e","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d240652dd1272d3b8c94d44e: done: 0.015062 sec","job_status":"done","queue_duration_s":3.713681,"scheduling_latency_s":3.713697,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015062,"completed_at":"2026-08-28T11:40:10.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 310.044255] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae20ce679b9e223e0cf48973","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae20ce679b9e223e0cf48973: start","job_status":"start","queue_duration_s":3.729903,"scheduling_latency_s":3.72992}1546gitlab # [ 310.056509] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae20ce679b9e223e0cf48973","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae20ce679b9e223e0cf48973: done: 0.01482 sec","job_status":"done","queue_duration_s":3.729903,"scheduling_latency_s":3.72992,"redis_calls":13,"redis_duration_s":0.001442,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01482,"completed_at":"2026-08-28T11:40:10.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 310.095310] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d616dd91ec1a5a40b2f81829","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d616dd91ec1a5a40b2f81829: start","job_status":"start","queue_duration_s":3.745857,"scheduling_latency_s":3.745874}1548gitlab # [ 310.107719] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d616dd91ec1a5a40b2f81829","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d616dd91ec1a5a40b2f81829: done: 0.014837 sec","job_status":"done","queue_duration_s":3.745857,"scheduling_latency_s":3.745874,"redis_calls":13,"redis_duration_s":0.001606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014837,"completed_at":"2026-08-28T11:40:10.858Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 310.147635] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2fd67beecffdf4974d3b65a","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2fd67beecffdf4974d3b65a: start","job_status":"start","queue_duration_s":3.762743,"scheduling_latency_s":3.762759}1550gitlab # [ 310.161363] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2fd67beecffdf4974d3b65a","created_at":"2026-08-28T11:40:06.541Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-66a80b580ade4b5f","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"203d560d4077c44717a30a61797d769a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.098Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2fd67beecffdf4974d3b65a: done: 0.014474 sec","job_status":"done","queue_duration_s":3.762743,"scheduling_latency_s":3.762759,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014474,"completed_at":"2026-08-28T11:40:10.875Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 310.200571] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:10.876Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"59582e7cb4ded0e0c2df0fcc","created_at":"2026-08-28T11:40:07.101Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69d9dd2aae64b000d970f29e56244c0d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.184Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-59582e7cb4ded0e0c2df0fcc: start","job_status":"start","queue_duration_s":3.692591,"scheduling_latency_s":3.692608}1552gitlab # [ 310.214112] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.130Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"59582e7cb4ded0e0c2df0fcc","created_at":"2026-08-28T11:40:07.101Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69d9dd2aae64b000d970f29e56244c0d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.184Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-59582e7cb4ded0e0c2df0fcc: done: 0.254143 sec","job_status":"done","queue_duration_s":3.692591,"scheduling_latency_s":3.692608,"redis_calls":10,"redis_duration_s":0.0013449999999999998,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024852,"duration_s":0.254143,"completed_at":"2026-08-28T11:40:11.130Z","load_balancing_strategy":"primary","db_duration_s":0.001205,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 310.255256] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c8efdf31e3bc1d7f6c534b47","created_at":"2026-08-28T11:40:07.193Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1d48f9b2ee9d3a4b0a0dfb0489d7a56","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.199Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c8efdf31e3bc1d7f6c534b47: start","job_status":"start","queue_duration_s":3.934799,"scheduling_latency_s":3.934847}1554gitlab # [ 310.269900] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.189Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c8efdf31e3bc1d7f6c534b47","created_at":"2026-08-28T11:40:07.193Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1d48f9b2ee9d3a4b0a0dfb0489d7a56","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.199Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c8efdf31e3bc1d7f6c534b47: done: 0.055422 sec","job_status":"done","queue_duration_s":3.934799,"scheduling_latency_s":3.934847,"redis_calls":12,"redis_duration_s":0.005174,"redis_read_bytes":264,"redis_write_bytes":1466,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":227,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002792,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001655,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0115955,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.055422,"completed_at":"2026-08-28T11:40:11.189Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004044,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1555gitlab # [ 310.313875] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"5b80555932fb18742f25bb97","created_at":"2026-08-28T11:40:07.211Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"120ba20b25c2efb625b117eb4fa061cd","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.220Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-5b80555932fb18742f25bb97: start","job_status":"start","queue_duration_s":3.972204,"scheduling_latency_s":3.972244}1556gitlab # [ 310.327346] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"5b80555932fb18742f25bb97","created_at":"2026-08-28T11:40:07.211Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"120ba20b25c2efb625b117eb4fa061cd","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.220Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-5b80555932fb18742f25bb97: done: 0.08905 sec","job_status":"done","queue_duration_s":3.972204,"scheduling_latency_s":3.972244,"redis_calls":10,"redis_duration_s":0.0030239999999999998,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068522,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00522458},"duration_s":0.08905,"completed_at":"2026-08-28T11:40:11.281Z","load_balancing_strategy":"primary","db_duration_s":0.023252,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 310.371437] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e60c5832e8b01c132368e006","created_at":"2026-08-28T11:40:07.231Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddfc41582a93aa141edfadb139bc0a80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-28T11:40:07.238Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e60c5832e8b01c132368e006: start","job_status":"start","queue_duration_s":4.045247,"scheduling_latency_s":4.045286}1558gitlab # [ 310.385021] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.302Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1559gitlab # [ 310.388230] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.311Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e60c5832e8b01c132368e006","created_at":"2026-08-28T11:40:07.231Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddfc41582a93aa141edfadb139bc0a80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-28T11:40:07.238Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e60c5832e8b01c132368e006: done: 0.027217 sec","job_status":"done","queue_duration_s":4.045247,"scheduling_latency_s":4.045286,"redis_calls":9,"redis_duration_s":0.002365,"redis_read_bytes":261,"redis_write_bytes":1049,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000588,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":243,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027217,"completed_at":"2026-08-28T11:40:11.311Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560gitlab # [ 310.429384] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"613367046995068e309234ab","created_at":"2026-08-28T11:40:07.241Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db47f3dd3f2a3af63ef1bd4ebc7bc09","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.247Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-613367046995068e309234ab: start","job_status":"start","queue_duration_s":4.065396,"scheduling_latency_s":4.065425}1561gitlab # [ 310.443751] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.325Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1562gitlab # [ 310.447535] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"613367046995068e309234ab","created_at":"2026-08-28T11:40:07.241Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db47f3dd3f2a3af63ef1bd4ebc7bc09","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.247Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-613367046995068e309234ab: done: 0.018574 sec","job_status":"done","queue_duration_s":4.065396,"scheduling_latency_s":4.065425,"redis_calls":9,"redis_duration_s":0.001488,"redis_read_bytes":261,"redis_write_bytes":995,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000333,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":229,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018574,"completed_at":"2026-08-28T11:40:11.331Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 310.489744] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d3e4ac6476d8439668f29725","created_at":"2026-08-28T11:40:07.251Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3514f00d87d4e943d0650b55490049dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-28T11:40:07.263Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d3e4ac6476d8439668f29725: start","job_status":"start","queue_duration_s":4.070948,"scheduling_latency_s":4.070969}1564gitlab # [ 310.503674] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.345Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1565gitlab # [ 310.507749] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d3e4ac6476d8439668f29725","created_at":"2026-08-28T11:40:07.251Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3514f00d87d4e943d0650b55490049dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-28T11:40:07.263Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d3e4ac6476d8439668f29725: done: 0.017978 sec","job_status":"done","queue_duration_s":4.070948,"scheduling_latency_s":4.070969,"redis_calls":9,"redis_duration_s":0.00333,"redis_read_bytes":261,"redis_write_bytes":1042,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":241,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017978,"completed_at":"2026-08-28T11:40:11.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566gitlab # [ 310.549333] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.353Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"09ba960fb6b9a837b62d3456","created_at":"2026-08-28T11:40:07.270Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f0528310a06b152073c0229969fc87a","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.284Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-09ba960fb6b9a837b62d3456: start","job_status":"start","queue_duration_s":4.068955,"scheduling_latency_s":4.068973}1567gitlab # [ 310.564843] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.383Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"09ba960fb6b9a837b62d3456","created_at":"2026-08-28T11:40:07.270Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f0528310a06b152073c0229969fc87a","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.284Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-09ba960fb6b9a837b62d3456: done: 0.030877 sec","job_status":"done","queue_duration_s":4.068955,"scheduling_latency_s":4.068973,"redis_calls":10,"redis_duration_s":0.001396,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026072,"duration_s":0.030877,"completed_at":"2026-08-28T11:40:11.383Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568gitlab # [ 310.609384] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.385Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"33a9a95d8e441c09e4296a0f","created_at":"2026-08-28T11:40:07.288Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20eb33fae8722d4c989d096c5a489881","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.303Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-33a9a95d8e441c09e4296a0f: start","job_status":"start","queue_duration_s":4.081899,"scheduling_latency_s":4.081915}1569gitlab # [ 310.622714] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.409Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"33a9a95d8e441c09e4296a0f","created_at":"2026-08-28T11:40:07.288Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20eb33fae8722d4c989d096c5a489881","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.303Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-33a9a95d8e441c09e4296a0f: done: 0.024205 sec","job_status":"done","queue_duration_s":4.081899,"scheduling_latency_s":4.081915,"redis_calls":10,"redis_duration_s":0.001487,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027676,"duration_s":0.024205,"completed_at":"2026-08-28T11:40:11.409Z","load_balancing_strategy":"primary","db_duration_s":0.000646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570gitlab # [ 310.663823] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"890726497f2c6eb305dd9e2f","created_at":"2026-08-28T11:40:07.305Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61c16d21d2d0db67ecb5134cc7ea4514","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-890726497f2c6eb305dd9e2f: start","job_status":"start","queue_duration_s":4.101664,"scheduling_latency_s":4.101682}1571gitlab # [ 310.678994] postgres[861]: [861] LOG: checkpoint starting: time1572gitlab # [ 310.679868] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"890726497f2c6eb305dd9e2f","created_at":"2026-08-28T11:40:07.305Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61c16d21d2d0db67ecb5134cc7ea4514","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.308Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-890726497f2c6eb305dd9e2f: done: 0.04226 sec","job_status":"done","queue_duration_s":4.101664,"scheduling_latency_s":4.101682,"redis_calls":9,"redis_duration_s":0.0012699999999999999,"redis_read_bytes":261,"redis_write_bytes":1009,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":233,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04226,"completed_at":"2026-08-28T11:40:11.452Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573gitlab # [ 310.722357] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5383ce161517e5c222a072e7","created_at":"2026-08-28T11:40:07.356Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98b16629cdbd63a96e6f9d045f77e15d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.366Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5383ce161517e5c222a072e7: start","job_status":"start","queue_duration_s":4.087872,"scheduling_latency_s":4.087891}1574gitlab # [ 310.736904] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5383ce161517e5c222a072e7","created_at":"2026-08-28T11:40:07.356Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98b16629cdbd63a96e6f9d045f77e15d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.366Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5383ce161517e5c222a072e7: done: 0.024961 sec","job_status":"done","queue_duration_s":4.087872,"scheduling_latency_s":4.087891,"redis_calls":12,"redis_duration_s":0.001619,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005219,"exclusive_lock_hold_duration_s":0.00688388,"duration_s":0.024961,"completed_at":"2026-08-28T11:40:11.479Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1575gitlab # [ 310.780820] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fe943dba070917c41456368b","created_at":"2026-08-28T11:40:07.373Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb753076bfdcd17436745c5cefecab12","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.378Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fe943dba070917c41456368b: start","job_status":"start","queue_duration_s":4.101936,"scheduling_latency_s":4.101952}1576gitlab # [ 310.794951] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fe943dba070917c41456368b","created_at":"2026-08-28T11:40:07.373Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-dca83036cb8d4a54","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb753076bfdcd17436745c5cefecab12","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:07.378Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fe943dba070917c41456368b: done: 0.307671 sec","job_status":"done","queue_duration_s":4.101936,"scheduling_latency_s":4.101952,"redis_calls":564,"redis_duration_s":0.054790000000000005,"redis_read_bytes":816,"redis_write_bytes":48953,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":193,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.053775,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.271201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.307671,"completed_at":"2026-08-28T11:40:11.787Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577gitlab # [ 310.837459] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.789Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"583f0e081544cc3cbde9342c","created_at":"2026-08-28T11:40:09.614Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-f5696e580acd42eb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1e4b9957d3e1a5c413e7630ccf174d53","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:09.691Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-583f0e081544cc3cbde9342c: start","job_status":"start","queue_duration_s":2.097745,"scheduling_latency_s":2.097769}1578gitlab # [ 310.882666] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.821Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"583f0e081544cc3cbde9342c","created_at":"2026-08-28T11:40:09.614Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-f5696e580acd42eb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1e4b9957d3e1a5c413e7630ccf174d53","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:09.691Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-583f0e081544cc3cbde9342c: done: 0.031906 sec","job_status":"done","queue_duration_s":2.097745,"scheduling_latency_s":2.097769,"redis_calls":10,"redis_duration_s":0.002176,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037674,"duration_s":0.031906,"completed_at":"2026-08-28T11:40:11.821Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579gitlab # [ 310.937358] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.822Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"70abc476609529448c519bfe","created_at":"2026-08-28T11:40:09.693Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-f5696e580acd42eb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1e4b9957d3e1a5c413e7630ccf174d53","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:09.703Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-70abc476609529448c519bfe: start","job_status":"start","queue_duration_s":2.118626,"scheduling_latency_s":2.118646}1580gitlab # [ 310.952669] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:11.844Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"70abc476609529448c519bfe","created_at":"2026-08-28T11:40:09.693Z","trace_propagation_headers":{"sentry-trace":"baef5b09a481492da7047c7970996ed2-f5696e580acd42eb","baggage":"sentry-trace_id=baef5b09a481492da7047c7970996ed2,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1e4b9957d3e1a5c413e7630ccf174d53","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:09.703Z","job_size_bytes":2,"pid":2043,"sidekiq_tid":"mao7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-70abc476609529448c519bfe: done: 0.021933 sec","job_status":"done","queue_duration_s":2.118626,"scheduling_latency_s":2.118646,"redis_calls":10,"redis_duration_s":0.00143,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002458,"duration_s":0.021933,"completed_at":"2026-08-28T11:40:11.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581gitlab # [ 312.029837] gitaly[907]: time="2026-08-28T11:40:14.921Z" level=info msg="burden monitor stats" pid=9071582gitlab # [ 329.154633] postgres[3400]: [3400] ERROR: relation "table_without_models" does not exist1583gitlab # [ 329.158214] postgres[3400]: [3400] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1584gitlab # [ 335.429481] runuser[3053]: pam_unix(runuser:session): session closed for user gitlab1585gitlab # [ 335.431350] setup-gitlab-start[3027]: + systemctl restart gitlab-sidekiq.service1586gitlab # [ 335.479880] systemd[1]: Stopping gitlab-sidekiq.service...1587gitlab # [ 335.483661] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:38.372Z","message":"Shutting down"}1588gitlab # [ 335.488111] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:38.372Z","message":"Scheduler exiting..."}1589gitlab # [ 335.492327] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:38.373Z","message":"Terminating quiet threads for default capsule"}1590gitlab # [ 335.497412] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:38.373Z","message":"Scheduler exiting..."}1591gitlab # [ 335.981315] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:38.874Z","message":"Pausing to allow jobs to finish..."}1592gitlab # [ 339.488408] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:42.380Z","message":"Bye!"}1593gitlab # [ 339.533356] sidekiq-cluster[2043]: {"severity":"INFO","time":"2026-08-28T11:40:42.426Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2043,"worker_id":"sidekiq_0","memwd_rss_bytes":935395328,"retry":0}1594gitlab # [ 340.831133] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1595gitlab # [ 340.831563] systemd[1]: Stopped gitlab-sidekiq.service.1596gitlab # [ 340.831774] systemd[1]: gitlab-sidekiq.service: Consumed 1min 59.776s CPU time over 3min 34.865s wall clock time, 935.5M memory peak, 4.5M read from disk.1597gitlab # [ 340.845298] systemd[1]: Started gitlab-sidekiq.service.1598gitlab # [ 340.849927] setup-gitlab-start[3027]: + mkdir -p /var/lib/secrets1599gitlab # [ 340.855558] setup-gitlab-start[3027]: + echo nixbot-test-token-of-20-chars1600gitlab # [ 340.859201] setup-gitlab-start[3027]: + chmod 644 /var/lib/secrets/gitlab-token1601gitlab # [ 340.862424] setup-gitlab-start[3027]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1602gitlab # [ 342.068249] gitaly[907]: time="2026-08-28T11:40:44.960Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:54.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:44.960" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 342.091960] gitaly[907]: time="2026-08-28T11:40:44.961Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:54.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:44.960" grpc.time_ms=1.194 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604gitlab # [ 342.307737] gitaly[907]: time="2026-08-28T11:40:45.201Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.200" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1605gitlab # [ 342.368466] gitaly[907]: time="2026-08-28T11:40:45.201Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.200" grpc.time_ms=0.604 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1606gitlab # [ 342.443901] gitaly[907]: time="2026-08-28T11:40:45.337Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX diskcache=21117ded-ad70-436c-8768-03a6f913eac2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-28T11:40:55.085" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.276" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 342.472951] gitaly[907]: time="2026-08-28T11:40:45.338Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=33 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1441 command.oublock=32 command.real_time_ms=39 command.spawn_token_fork_ms=2 command.system_time_ms=27 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-28T11:40:55.085" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.276" grpc.time_ms=61.597 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1608gitlab # [ 342.499851] gitaly[907]: time="2026-08-28T11:40:45.355Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.163" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.354" pid=907 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1609gitlab # [ 342.518981] gitaly[907]: time="2026-08-28T11:40:45.355Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.163" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.354" grpc.time_ms=1.152 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 342.535967] gitaly[907]: time="2026-08-28T11:40:45.375Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-28T11:40:55.171" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.362" grpc.time_ms=12.621 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1611gitlab # [ 342.576900] gitaly[907]: time="2026-08-28T11:40:45.470Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.279" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.470" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1612gitlab # [ 342.589694] gitaly[907]: time="2026-08-28T11:40:45.470Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.279" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.470" grpc.time_ms=0.442 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1613gitlab # [ 342.627399] gitaly[907]: time="2026-08-28T11:40:45.521Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX diskcache=530d15a1-aecb-44a7-ae0c-8712fd834170 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-28T11:40:55.283" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.473" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 342.641768] gitaly[907]: time="2026-08-28T11:40:45.521Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=29 command.spawn_token_fork_ms=1 command.system_time_ms=22 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-28T11:40:55.283" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.473" grpc.time_ms=47.385 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615gitlab # [ 342.660181] gitaly[907]: time="2026-08-28T11:40:45.530Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.530" pid=907 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1616gitlab # [ 342.672839] gitaly[907]: time="2026-08-28T11:40:45.530Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.530" grpc.time_ms=0.637 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1617gitlab # [ 342.685555] gitaly[907]: time="2026-08-28T11:40:45.552Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=9 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-28T11:40:55.016" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.537" grpc.time_ms=14.367 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1618gitlab # [ 342.708350] gitaly[907]: time="2026-08-28T11:40:45.602Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:40:55.081" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.601" grpc.time_ms=0.217 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1619gitlab # [ 342.728412] gitaly[907]: time="2026-08-28T11:40:45.611Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-28T11:40:55.090" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:45.610" grpc.time_ms=0.341 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1620gitlab # [ 343.468458] gitaly[907]: time="2026-08-28T11:40:46.360Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M142R5C6Q34JMSEZ5FGZ6YMX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-28T11:40:56.152" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:46.342" grpc.time_ms=18.01 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1621gitlab # [ 344.105623] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:46 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 32161622gitlab # [ 344.110047] setup-gitlab-start[3027]: + mkdir -p /tmp/test-flake1623gitlab # [ 344.115101] setup-gitlab-start[3027]: + cd /tmp/test-flake1624gitlab # [ 344.115971] setup-gitlab-start[3027]: + git init -b master1625gitlab # [ 344.143342] setup-gitlab-start[3524]: Initialized empty Git repository in /tmp/test-flake/.git/1626gitlab # [ 344.144924] setup-gitlab-start[3027]: + git config user.name test1627gitlab # [ 344.151654] setup-gitlab-start[3027]: + git config user.email test@example.com1628gitlab # [ 344.158741] setup-gitlab-start[3027]: + cat1629gitlab # [ 344.164371] setup-gitlab-start[3027]: + git add flake.nix1630gitlab # [ 344.173148] setup-gitlab-start[3027]: + git commit -m 'initial commit'1631gitlab # [ 344.200787] setup-gitlab-start[3529]: [master (root-commit) 39ac7f2] initial commit1632gitlab # [ 344.203925] setup-gitlab-start[3529]: 1 file changed, 10 insertions(+)1633gitlab # [ 344.207552] setup-gitlab-start[3529]: create mode 100644 flake.nix1634gitlab # [ 344.210429] setup-gitlab-start[3027]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1635gitlab # [ 344.215210] setup-gitlab-start[3027]: + git push -u origin master1636gitlab # [ 344.329520] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:47 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 741637gitlab # [ 345.443885] gitaly[907]: time="2026-08-28T11:40:48.336Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142R8QRBEVA6NJRXRFWCZPF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:48.318" grpc.time_ms=17.818 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1638gitlab # [ 345.483379] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:48 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 11121639gitlab # [ 345.890860] gitaly[907]: time="2026-08-28T11:40:48.784Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-28T11:40:58.246" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:48.766" grpc.time_ms=17.582 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1640gitlab # [ 345.929937] sidekiq-cluster[3471]: {"severity":"INFO","time":"2026-08-28T11:40:48.786Z","message":"Starting cluster with 1 processes"}1641gitlab # [ 345.972928] gitaly[907]: time="2026-08-28T11:40:48.866Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=8 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-28T11:41:18.052" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-28T11:40:48.852" grpc.time_ms=13.859 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1642gitlab # [ 346.050728] gitaly[907]: time="2026-08-28T11:40:48.944Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M142R9WA3FDZ5899KKJJ6494 duration_ms=317ns method=POST pid=907 status=200 url="http://unix/api/v4/internal/allowed"1643gitlab # [ 346.141521] gitaly[907]: time="2026-08-28T11:40:49.035Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M142R9WA3FDZ5899KKJJ6494 duration_ms=89ns method=POST pid=907 status=200 url="http://unix/api/v4/internal/pre_receive"1644gitlab # [ 346.154274] gitaly[907]: time="2026-08-28T11:40:49.035Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5314 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:48.624" grpc.time_ms=411.485 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1645gitlab # [ 346.239545] gitaly[907]: time="2026-08-28T11:40:49.133Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:49.130" grpc.time_ms=1.906 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1646gitlab # [ 346.330193] gitaly[907]: time="2026-08-28T11:40:49.223Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:49.222" grpc.time_ms=1.164 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1647gitlab # [ 346.411580] gitaly[907]: time="2026-08-28T11:40:49.305Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:49.303" grpc.time_ms=0.959 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1648gitlab # [ 346.763420] gitaly[907]: time="2026-08-28T11:40:49.656Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-28T11:40:59.011" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:49.641" grpc.time_ms=14.517 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1649gitlab # [ 346.834414] gitaly[907]: time="2026-08-28T11:40:49.728Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=25 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=5 command.system_time_ms=25 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-28T11:40:59.055" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:40:49.686" grpc.time_ms=41.11 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1650gitlab # [ 346.881964] gitaly[907]: time="2026-08-28T11:40:49.775Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M142R9WA3FDZ5899KKJJ6494 duration_ms=394ns method=POST pid=907 status=200 url="http://unix/api/v4/internal/post_receive"1651gitlab # [ 346.890362] gitaly[907]: time="2026-08-28T11:40:49.777Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4808 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:49.380" grpc.time_ms=396.652 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1652gitlab # [ 346.911040] gitaly[907]: time="2026-08-28T11:40:49.791Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 diskcache=c9027db3-e47a-4b95-90ea-826ef07e68d7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:48.510" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1653gitlab # [ 346.927797] setup-gitlab-start[3533]: To http://gitlab/root/test-flake.git1654gitlab # [ 346.929260] setup-gitlab-start[3533]: * [new branch] master -> master1655gitlab # [ 346.930567] setup-gitlab-start[3533]: branch 'master' set up to track 'origin/master'.1656gitlab # [ 346.932575] gitlab-workhorse[820]: time="2026-08-28T11:40:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521657gitlab # [ 346.935093] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:49 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 13971658gitlab # [ 346.938199] systemd[1]: Finished setup-gitlab.service.1659gitlab # [ 346.939199] gitaly[907]: time="2026-08-28T11:40:49.791Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=356 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=13180 command.oublock=32 command.real_time_ms=1272 command.spawn_token_fork_ms=1 command.system_time_ms=176 command.user_time_ms=180 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:48.510" grpc.time_ms=1280.734 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1660gitlab # [ 346.956181] systemd[1]: Starting nixbot CI service...1661gitlab: (finished: waiting for unit setup-gitlab.service, in 348.53 seconds)1662gitlab: waiting for unit nixbot.service1663gitlab # [ 348.092251] nixbot[3630]: {"ts": "2026-08-28T11:40:50.984Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1664gitlab # [ 348.132961] nixbot[3630]: {"ts": "2026-08-28T11:40:51.026Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1665gitlab # [ 348.152985] nixbot[3630]: {"ts": "2026-08-28T11:40:51.046Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1666gitlab # [ 348.159770] nixbot[3630]: {"ts": "2026-08-28T11:40:51.051Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1667gitlab # [ 348.166737] nixbot[3630]: {"ts": "2026-08-28T11:40:51.055Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1668gitlab # [ 348.173257] nixbot[3630]: {"ts": "2026-08-28T11:40:51.061Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1669gitlab # [ 348.179846] nixbot[3630]: {"ts": "2026-08-28T11:40:51.065Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1670gitlab # [ 348.186565] nixbot[3630]: {"ts": "2026-08-28T11:40:51.068Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1671gitlab # [ 348.193518] nixbot[3630]: {"ts": "2026-08-28T11:40:51.073Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1672gitlab # [ 348.200332] nixbot[3630]: {"ts": "2026-08-28T11:40:51.076Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1673gitlab # [ 348.208206] nixbot[3630]: {"ts": "2026-08-28T11:40:51.081Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1674gitlab # [ 348.214204] nixbot[3630]: {"ts": "2026-08-28T11:40:51.086Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1675gitlab # [ 348.220393] nixbot[3630]: {"ts": "2026-08-28T11:40:51.090Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1676gitlab # [ 348.225956] nixbot[3630]: {"ts": "2026-08-28T11:40:51.092Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1677gitlab # [ 348.232049] nixbot[3630]: {"ts": "2026-08-28T11:40:51.096Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1678gitlab # [ 348.237805] nixbot[3630]: {"ts": "2026-08-28T11:40:51.098Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1679gitlab # [ 348.242957] nixbot[3630]: {"ts": "2026-08-28T11:40:51.099Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1680gitlab # [ 348.247894] nixbot[3630]: {"ts": "2026-08-28T11:40:51.100Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1681gitlab # [ 348.252321] nixbot[3630]: {"ts": "2026-08-28T11:40:51.104Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1682gitlab # [ 348.256972] nixbot[3630]: {"ts": "2026-08-28T11:40:51.106Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1683gitlab # [ 348.268112] nixbot[3630]: {"ts": "2026-08-28T11:40:51.108Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1684gitlab # [ 348.271505] nixbot[3630]: {"ts": "2026-08-28T11:40:51.118Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1685gitlab # [ 348.275196] nixbot[3630]: {"ts": "2026-08-28T11:40:51.120Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1686gitlab # [ 348.278741] nixbot[3630]: {"ts": "2026-08-28T11:40:51.123Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1687gitlab # [ 348.282178] nixbot[3630]: {"ts": "2026-08-28T11:40:51.124Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1688gitlab # [ 348.285621] nixbot[3630]: {"ts": "2026-08-28T11:40:51.128Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1689gitlab # [ 348.733521] nixbot[3630]: INFO: Started server process [3630]1690gitlab # [ 348.736788] nixbot[3630]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1691gitlab # [ 348.754336] nixbot[3630]: INFO: ::ffff:127.0.0.1:55142 - "GET /health HTTP/1.1" 200 OK1692gitlab # [ 348.762036] systemd[1]: Started nixbot CI service.1693gitlab # [ 348.767739] systemd[1]: Reached target Multi-User System.1694gitlab # [ 348.771859] systemd[1]: Startup finished in 745ms (kernel) + 4.245s (initrd) + 5min 43.780s (userspace) = 5min 48.771s.1695gitlab: (finished: waiting for unit nixbot.service, in 1.09 seconds)1696gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1697gitlab # [ 349.225691] nixbot[3630]: INFO: ::ffff:127.0.0.1:55158 - "GET /health HTTP/1.1" 200 OK1698gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1699subtest: project discovered and webhook registered1700gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1701gitlab # [ 349.391195] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 921702gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)1703gitlab # [ 349.853092] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:52 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 10991704gitlab # [ 349.856538] nixbot[3630]: {"ts": "2026-08-28T11:40:52.749Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1705gitlab # [ 349.882097] nixbot[3630]: {"ts": "2026-08-28T11:40:52.775Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1706gitlab # [ 350.048986] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1501707gitlab # [ 350.053747] nixbot[3630]: {"ts": "2026-08-28T11:40:52.945Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1708gitlab # [ 350.058738] nixbot[3630]: {"ts": "2026-08-28T11:40:52.948Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1709gitlab # [ 350.367716] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 3071710gitlab # [ 350.381454] nixbot[3630]: {"ts": "2026-08-28T11:40:53.269Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1711gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1712gitlab # [ 350.548329] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 991713gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.15 seconds)1714(finished: subtest: project discovered and webhook registered, in 1.32 seconds)1715subtest: push delivers the webhook1716gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1717gitlab # [ 350.601254] nixbot[3630]: {"ts": "2026-08-28T11:40:53.494Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1718gitlab # [ 350.604170] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2081719gitlab # [ 350.683044] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 581720gitlab # [ 350.726199] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 1001721gitlab # [ 350.729909] nixbot[3630]: {"ts": "2026-08-28T11:40:53.623Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1722gitlab # [ 350.813997] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 751723gitlab # [ 350.816796] nixbot[3630]: {"ts": "2026-08-28T11:40:53.710Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1724gitlab # [ 350.819730] gitaly[907]: time="2026-08-28T11:40:53.710Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142REYAD6V6V7DTBQ3Z443Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:53.697" grpc.time_ms=13.304 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1725gitlab # Enumerating objects: 5, done.1726gitlab # [ 350.836430] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 1321727gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1728gitlab # Delta compression using up to 4 threads1729gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1730gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 68.00 KiB/s, done.1731gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1732gitlab # [ 350.900097] gitaly[907]: time="2026-08-28T11:40:53.790Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RF2NF8DJS3RK0V2Z661K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-08-28T11:41:23.075" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:40:53.776" grpc.time_ms=12.988 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1733gitlab # [ 350.918800] gitaly[907]: time="2026-08-28T11:40:53.807Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RF2NF8DJS3RK0V2Z661K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:41:23.104" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:40:53.805" grpc.time_ms=1.72 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1734gitlab # [ 350.940162] gitaly[907]: time="2026-08-28T11:40:53.833Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=7 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M142RF2NF8DJS3RK0V2Z661K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-08-28T11:41:03.182" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:40:53.813" grpc.time_ms=20.466 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1735gitlab # [ 350.976346] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1481736gitlab # [ 350.983127] nixbot[3630]: {"ts": "2026-08-28T11:40:53.866Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1737gitlab # [ 351.183091] gitaly[907]: time="2026-08-28T11:40:54.076Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-28T11:41:24.055" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-28T11:40:54.056" grpc.time_ms=20.431 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738gitlab # [ 351.239826] gitaly[907]: time="2026-08-28T11:40:54.133Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G duration_ms=167ns method=POST pid=907 status=200 url="http://unix/api/v4/internal/allowed"1739gitlab # [ 351.279243] gitaly[907]: time="2026-08-28T11:40:54.172Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G duration_ms=38ns method=POST pid=907 status=200 url="http://unix/api/v4/internal/pre_receive"1740gitlab # [ 351.290136] gitaly[907]: time="2026-08-28T11:40:54.173Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5314 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:53.965" grpc.time_ms=207.928 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1741gitlab # [ 351.321329] nixbot[3630]: {"ts": "2026-08-28T11:40:54.189Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1742gitlab # [ 351.324381] nixbot[3630]: {"ts": "2026-08-28T11:40:54.194Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1743gitlab # [ 351.326954] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 3221744gitlab # [ 351.337321] gitaly[907]: time="2026-08-28T11:40:54.231Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:54.230" grpc.time_ms=0.631 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1745gitlab # [ 351.390356] gitaly[907]: time="2026-08-28T11:40:54.283Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:54.282" grpc.time_ms=0.765 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1746gitlab # [ 351.430828] gitaly[907]: time="2026-08-28T11:40:54.324Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:54.324" grpc.time_ms=0.462 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1747gitlab # [ 351.450985] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 851748gitlab # [ 351.453598] gitaly[907]: time="2026-08-28T11:40:54.344Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=12 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M142RFKKJ2DFWA63FBS605J8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.317" grpc.time_ms=26.757 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1749gitlab # [ 351.555678] gitaly[907]: time="2026-08-28T11:40:54.448Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G duration_ms=81ns method=POST pid=907 status=200 url="http://unix/api/v4/internal/post_receive"1750gitlab # [ 351.563088] gitaly[907]: time="2026-08-28T11:40:54.456Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4808 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:54.366" grpc.time_ms=89.252 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1751gitlab # [ 351.582046] gitaly[907]: time="2026-08-28T11:40:54.469Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G diskcache=0657fc63-b3ee-4a0c-918a-0e92e4b78e54 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:53.885" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1752gitlab # To http://gitlab/root/test-flake.git1753gitlab # 39ac7f2..f1bc3ee master -> master1754gitlab # [ 351.598327] gitlab-workhorse[820]: time="2026-08-28T11:40:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521755gitlab # [ 351.604969] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7131756gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.07 seconds)1757gitlab: must succeed: git -C /tmp/test-flake rev-parse master1758gitlab # [ 351.607736] gitaly[907]: time="2026-08-28T11:40:54.470Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=236 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13179 command.oublock=32 command.real_time_ms=577 command.spawn_token_fork_ms=0 command.system_time_ms=147 command.user_time_ms=89 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:53.885" grpc.time_ms=584.593 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1759gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1760gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1761gitlab # [ 351.631685] gitaly[907]: time="2026-08-28T11:40:54.513Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142RFPBYQXNNQRZJVBFDSZ9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.500" pid=907 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1762gitlab # [ 351.645985] gitlab-workhorse[820]: time="2026-08-28T11:40:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131763gitlab # [ 351.648442] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1691764gitlab # [ 351.650813] gitaly[907]: time="2026-08-28T11:40:54.514Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M142RFPBYQXNNQRZJVBFDSZ9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.500" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1765gitlab # [ 351.666332] gitaly[907]: time="2026-08-28T11:40:54.518Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M142RFPBYQXNNQRZJVBFDSZ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.500" grpc.time_ms=14.93 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1766gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.11 seconds)1767gitlab # [ 351.740651] gitaly[907]: time="2026-08-28T11:40:54.634Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142RFVTHCJVQJ5RE2ZJZ36X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.613" pid=907 remote_ip="2001:db8:1::1" request_sha=e5cf5213fa2c70dd9435de4dd6eb02ec64a8296a response_bytes=417 span.kind=server system=grpc1768gitlab # [ 351.761609] gitaly[907]: time="2026-08-28T11:40:54.634Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M142RFVTHCJVQJ5RE2ZJZ36X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.613" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1769gitlab # [ 351.785689] nixbot[3630]: INFO: ::ffff:127.0.0.1:53442 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1770gitlab # [ 351.788661] gitlab-workhorse[820]: time="2026-08-28T11:40:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=4171771gitlab # [ 351.791290] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 417 "" "git/2.55.0" 1121772gitlab # [ 351.794076] gitaly[907]: time="2026-08-28T11:40:54.634Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M142RFVTHCJVQJ5RE2ZJZ36X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.613" grpc.time_ms=20.883 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1773gitlab # [ 351.849410] gitaly[907]: time="2026-08-28T11:40:54.742Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M142RG0K20CJTDW0H4PHMT77 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.727" grpc.time_ms=14.619 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1774gitlab # [ 351.870603] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 741775gitlab # [ 351.941823] gitaly[907]: time="2026-08-28T11:40:54.835Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142RG311A0YSJSRTDGTW9MA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.816" pid=907 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1776gitlab # [ 351.970573] gitaly[907]: time="2026-08-28T11:40:54.835Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RG311A0YSJSRTDGTW9MA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.816" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1777gitlab # [ 351.996093] gitlab-workhorse[820]: time="2026-08-28T11:40:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671778gitlab # [ 351.998927] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 821779gitlab # [ 352.001925] gitaly[907]: time="2026-08-28T11:40:54.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RG311A0YSJSRTDGTW9MA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.816" grpc.time_ms=19.847 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1780gitlab # [ 352.072151] gitaly[907]: time="2026-08-28T11:40:54.965Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M142RG78G2T3Z6QA3J23Q017 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:54.947" grpc.time_ms=17.635 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 352.106997] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 771782gitlab # [ 352.180162] gitaly[907]: time="2026-08-28T11:40:55.073Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142RG9RK8BCY8RYS6XV97FR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:55.041" pid=907 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1783gitlab # [ 352.212226] gitaly[907]: time="2026-08-28T11:40:55.074Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M142RG9RK8BCY8RYS6XV97FR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:55.041" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 352.248663] gitlab-workhorse[820]: time="2026-08-28T11:40:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091785gitlab # [ 352.253113] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1061786gitlab # [ 352.257657] gitaly[907]: time="2026-08-28T11:40:55.074Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M142RG9RK8BCY8RYS6XV97FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:40:55.041" grpc.time_ms=33.17 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1787gitlab # [ 352.356235] gitaly[907]: time="2026-08-28T11:40:55.249Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:41:25.247" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:40:55.248" grpc.time_ms=0.959 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1788gitlab # [ 352.424870] gitaly[907]: time="2026-08-28T11:40:55.318Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-28T11:41:25.298" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:40:55.298" grpc.time_ms=19.267 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 352.470977] gitaly[907]: time="2026-08-28T11:40:55.364Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:41:05.158" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:40:55.348" grpc.time_ms=15.835 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 352.504282] gitaly[907]: time="2026-08-28T11:40:55.397Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:41:05.190" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:40:55.381" grpc.time_ms=16.239 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1791gitlab # [ 352.621380] gitaly[907]: time="2026-08-28T11:40:55.515Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-28T11:41:05.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:55.514" grpc.time_ms=0.605 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792gitlab # [ 352.657362] gitaly[907]: time="2026-08-28T11:40:55.550Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-28T11:41:05.021" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:40:55.532" grpc.time_ms=18.426 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1794gitlab # [ 352.731081] gitaly[907]: time="2026-08-28T11:40:55.624Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:55.624" grpc.time_ms=0.598 pid=907 span.kind=server system=grpc1795gitlab # [ 352.783527] gitaly[907]: time="2026-08-28T11:40:55.677Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:40:55.676" grpc.time_ms=0.948 pid=907 span.kind=server system=grpc1796gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.07 seconds)1797gitlab # [ 352.809909] gitaly[907]: time="2026-08-28T11:40:55.689Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H diskcache=6836fd63-4c23-442f-870b-195fec9817f6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-28T11:41:05.045" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:55.556" pid=907 ref=refs/keep-around/39ac7f2789de299302f12fa1c23545460e749303 remote_ip="2001:db8:1::1" revision=39ac7f2789de299302f12fa1c23545460e749303 span.kind=server system=grpc username=root1798gitlab # [ 352.833741] nixbot[3630]: INFO: ::ffff:127.0.0.1:53444 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1799gitlab # [ 352.837317] gitaly[907]: time="2026-08-28T11:40:55.690Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=122 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5359 command.oublock=8 command.real_time_ms=123 command.spawn_token_fork_ms=2 command.system_time_ms=56 command.user_time_ms=66 component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-28T11:41:05.045" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:40:55.556" grpc.time_ms=133.604 pid=907 ref=refs/keep-around/39ac7f2789de299302f12fa1c23545460e749303 remote_ip="2001:db8:1::1" revision=39ac7f2789de299302f12fa1c23545460e749303 span.kind=server system=grpc username=root1800gitlab # [ 352.862263] gitaly[907]: time="2026-08-28T11:40:55.696Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:41:25.695" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:40:55.695" grpc.time_ms=0.476 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 353.031193] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:40:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 7771802gitlab # [ 353.039422] nixbot[3630]: {"ts": "2026-08-28T11:40:55.925Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 \"HTTP/1.1 201 Created\""}1803gitlab # [ 353.049587] nixbot[3630]: {"ts": "2026-08-28T11:40:55.937Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-39ac7f2789de-0"}1804gitlab # [ 353.370729] nixbot[3630]: {"ts": "2026-08-28T11:40:56.264Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-39ac7f2789de-0", "argv0": "bwrap"}1805gitlab # [ 353.538846] systemd[1]: Started Nix Daemon.1806gitlab # [ 353.650250] nix-daemon[3856]: accepted connection from pid 3851, user nixbot1807gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1808gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1809gitlab # [ 353.869638] nixbot[3630]: INFO: ::ffff:127.0.0.1:53450 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1810gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1811gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1812gitlab # [ 354.924375] nixbot[3630]: INFO: ::ffff:127.0.0.1:53462 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1813gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1814gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1815gitlab # [ 355.978911] nixbot[3630]: INFO: ::ffff:127.0.0.1:53476 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1816gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1817gitlab # [ 357.036006] nixbot[3630]: INFO: ::ffff:127.0.0.1:53482 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1818gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1819gitlab # [ 357.859240] nix-daemon[3856]: accepted connection from pid 3881, user nixbot1820gitlab # [ 357.883144] nix-daemon[3856]: accepted connection from pid 3851, user nixbot1821gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1822gitlab # [ 358.092690] nixbot[3630]: INFO: ::ffff:127.0.0.1:53492 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1823gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1824gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1825gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1826gitlab # [ 359.150590] nixbot[3630]: INFO: ::ffff:127.0.0.1:53506 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1827gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1828gitlab # [ 360.204344] nixbot[3630]: INFO: ::ffff:127.0.0.1:53516 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1829gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1830gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1831gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1832gitlab # [ 361.260582] nixbot[3630]: INFO: ::ffff:127.0.0.1:55930 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1833gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1834gitlab # [ 362.312483] gitaly[907]: time="2026-08-28T11:41:05.201Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RT508PAWQQMM1K6H3NHP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:41:35.099" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:41:05.200" grpc.time_ms=1.297 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.08 seconds)1836gitlab # [ 362.316825] nixbot[3630]: INFO: ::ffff:127.0.0.1:55940 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1837gitlab # [ 362.617009] gitaly[907]: time="2026-08-28T11:41:05.510Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142RT508PAWQQMM1K6H3NHP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:41:15.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:41:05.491" grpc.time_ms=19.038 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 362.833208] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:41:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 6691839gitlab # [ 362.841324] nixbot[3630]: {"ts": "2026-08-28T11:41:05.727Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 \"HTTP/1.1 201 Created\""}1840gitlab # [ 362.912465] gitaly[907]: time="2026-08-28T11:41:05.806Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RTT3WF9RHHXAF2CS8F1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:41:35.103" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:41:05.804" grpc.time_ms=1.269 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841gitlab # [ 362.970485] gitaly[907]: time="2026-08-28T11:41:05.864Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RTT3WF9RHHXAF2CS8F1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:41:15.214" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:41:05.845" grpc.time_ms=18.592 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 363.049331] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:41:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2101843gitlab # [ 363.057685] nixbot[3630]: {"ts": "2026-08-28T11:41:05.944Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 \"HTTP/1.1 201 Created\""}1844gitlab # [ 363.177556] gitaly[907]: time="2026-08-28T11:41:06.071Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RV1FWSH8QRQK6Y9BRX26 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:41:36.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:41:06.069" grpc.time_ms=1.194 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 363.257984] gitaly[907]: time="2026-08-28T11:41:06.151Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RV1FWSH8QRQK6Y9BRX26 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:41:16.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:41:06.134" grpc.time_ms=17.233 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1847gitlab # [ 363.368803] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:41:06 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2951848gitlab # [ 363.375018] nixbot[3630]: {"ts": "2026-08-28T11:41:06.264Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 \"HTTP/1.1 201 Created\""}1849gitlab # [ 363.414464] nixbot[3630]: INFO: ::ffff:127.0.0.1:55950 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1850gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.08 seconds)1851gitlab # [ 363.745453] gitaly[907]: time="2026-08-28T11:41:06.638Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RVJD401ZWERK73N2HZST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:41:06.619" grpc.time_ms=18.504 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1852gitlab # [ 363.777905] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:41:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1291853gitlab # [ 363.898288] gitaly[907]: time="2026-08-28T11:41:06.791Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142RVPHGEQECFGC8RDGX818 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:41:06.771" pid=907 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1854gitlab # [ 363.934671] gitaly[907]: time="2026-08-28T11:41:06.792Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RVPHGEQECFGC8RDGX818 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:41:06.771" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1855gitlab # [ 363.974234] gitlab-workhorse[820]: time="2026-08-28T11:41:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671856gitlab # [ 363.978468] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:41:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1521857gitlab # [ 363.982431] gitaly[907]: time="2026-08-28T11:41:06.792Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RVPHGEQECFGC8RDGX818 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:41:06.771" grpc.time_ms=21.103 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc1858gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1859gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1860gitlab # [ 364.476302] nixbot[3630]: INFO: ::ffff:127.0.0.1:55964 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1861gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1862gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1863gitlab # [ 365.536100] nixbot[3630]: INFO: ::ffff:127.0.0.1:55980 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1864gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1865gitlab # [ 366.592342] nixbot[3630]: INFO: ::ffff:127.0.0.1:55984 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1866gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1867gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1868gitlab # [ 367.649869] nixbot[3630]: INFO: ::ffff:127.0.0.1:55996 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1869gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1870gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1871gitlab # [ 368.706461] nixbot[3630]: INFO: ::ffff:127.0.0.1:56004 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1872gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1873gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1874gitlab # [ 369.763467] nixbot[3630]: INFO: ::ffff:127.0.0.1:56014 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1875gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1876gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1877gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1878gitlab # [ 370.819824] nixbot[3630]: INFO: ::ffff:127.0.0.1:56026 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1879gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1880gitlab # [ 371.874120] nixbot[3630]: INFO: ::ffff:127.0.0.1:47620 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1881gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1882gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1883gitlab # [ 372.931294] nixbot[3630]: INFO: ::ffff:127.0.0.1:47630 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1884gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1885gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1886gitlab # [ 373.988348] nixbot[3630]: INFO: ::ffff:127.0.0.1:47640 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1887gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1888gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1889gitlab # [ 375.045596] nixbot[3630]: INFO: ::ffff:127.0.0.1:47656 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1890gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1891gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1892gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1893gitlab # [ 376.102555] nixbot[3630]: INFO: ::ffff:127.0.0.1:47662 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1894gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1895gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1896gitlab # [ 377.158354] nixbot[3630]: INFO: ::ffff:127.0.0.1:47670 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1897gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1898gitlab # [ 378.213208] nixbot[3630]: INFO: ::ffff:127.0.0.1:47682 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1899gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1900gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1901gitlab # [ 379.272089] nixbot[3630]: INFO: ::ffff:127.0.0.1:47692 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1902gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1903gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1904gitlab # [ 380.329791] nixbot[3630]: INFO: ::ffff:127.0.0.1:47698 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1905gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1906gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1907gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1908gitlab # [ 381.384716] nixbot[3630]: INFO: ::ffff:127.0.0.1:35016 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1909gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1910gitlab # [ 382.440150] nixbot[3630]: INFO: ::ffff:127.0.0.1:35024 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1911gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1912gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1913gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1914gitlab # [ 383.498420] nixbot[3630]: INFO: ::ffff:127.0.0.1:35030 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1915gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1916gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1917gitlab # [ 384.554093] nixbot[3630]: INFO: ::ffff:127.0.0.1:35044 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1918gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1919gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1920gitlab # [ 385.609386] nixbot[3630]: INFO: ::ffff:127.0.0.1:35052 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1921gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1922gitlab # [ 386.665140] nixbot[3630]: INFO: ::ffff:127.0.0.1:35056 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1923gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1924gitlab # [ 387.611546] sidekiq-cluster[3572]: 2026-08-28T11:41:30.503Z pid=3572 tid=yn0 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3572, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>427233280}1925gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1926gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1927gitlab # [ 387.721905] nixbot[3630]: INFO: ::ffff:127.0.0.1:35066 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1928gitlab # [ 387.908335] sidekiq-cluster[3572]: W, [2026-08-28T11:41:30.801829 #3572] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1929gitlab # [ 388.050114] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:41:30.943Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1930gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1931gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1932gitlab # [ 388.775542] nixbot[3630]: INFO: ::ffff:127.0.0.1:35068 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1933gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1934gitlab # [ 389.827385] nixbot[3630]: INFO: ::ffff:127.0.0.1:35070 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1935gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1936gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1937gitlab # [ 390.885219] nixbot[3630]: INFO: ::ffff:127.0.0.1:40584 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1938gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1939gitlab # [ 391.114026] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:41:34.006Z","message":"GitLab reliable fetch activated!"}1940gitlab # [ 391.119143] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:41:34.006Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1941gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1942gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1943gitlab # [ 391.942167] nixbot[3630]: INFO: ::ffff:127.0.0.1:40586 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1944gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1945gitlab # [ 392.996944] nixbot[3630]: INFO: ::ffff:127.0.0.1:40602 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1946gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1947gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1948gitlab # [ 394.051699] nixbot[3630]: INFO: ::ffff:127.0.0.1:40606 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1949gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1950gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1951gitlab # [ 395.109098] nixbot[3630]: INFO: ::ffff:127.0.0.1:40608 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1952gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1953gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1954gitlab # [ 396.166641] nixbot[3630]: INFO: ::ffff:127.0.0.1:40614 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1955gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1956gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1957gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)1958gitlab # [ 397.224330] nixbot[3630]: INFO: ::ffff:127.0.0.1:40622 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1959gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1960gitlab # [ 398.280165] nixbot[3630]: INFO: ::ffff:127.0.0.1:40628 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1961gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1962gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1963gitlab # [ 399.338298] nixbot[3630]: INFO: ::ffff:127.0.0.1:40638 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1964gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1965gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1966gitlab # [ 400.395783] nixbot[3630]: INFO: ::ffff:127.0.0.1:40642 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1967gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1968gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1969gitlab # [ 401.450997] nixbot[3630]: INFO: ::ffff:127.0.0.1:33732 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1970gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1971gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1972gitlab # [ 402.507146] nixbot[3630]: INFO: ::ffff:127.0.0.1:33746 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1973gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1974gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1975gitlab # [ 403.564267] nixbot[3630]: INFO: ::ffff:127.0.0.1:33754 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1976gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1977gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1978gitlab # [ 404.621061] nixbot[3630]: INFO: ::ffff:127.0.0.1:33766 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1979gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1980gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1981gitlab # [ 405.680002] nixbot[3630]: INFO: ::ffff:127.0.0.1:33778 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1982gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1983gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1984gitlab # [ 406.737215] nixbot[3630]: INFO: ::ffff:127.0.0.1:33782 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1985gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1986gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1987gitlab # [ 407.794743] nixbot[3630]: INFO: ::ffff:127.0.0.1:33788 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1988gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1989gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1990gitlab # [ 408.850557] nixbot[3630]: INFO: ::ffff:127.0.0.1:33790 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1991gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1992gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1993gitlab # [ 409.909500] nixbot[3630]: INFO: ::ffff:127.0.0.1:33796 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1994gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1995gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1996gitlab # [ 410.967290] nixbot[3630]: INFO: ::ffff:127.0.0.1:58472 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK1997gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)1998gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'1999gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)2000gitlab # [ 412.024485] nixbot[3630]: INFO: ::ffff:127.0.0.1:58486 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2001gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2002gitlab # [ 413.079891] nixbot[3630]: INFO: ::ffff:127.0.0.1:58498 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2003gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2004gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2005gitlab # [ 414.136615] nixbot[3630]: INFO: ::ffff:127.0.0.1:58502 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2006gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2007gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2008gitlab # [ 415.194618] nixbot[3630]: INFO: ::ffff:127.0.0.1:58518 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2009gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2010gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2011gitlab # [ 416.252159] nixbot[3630]: INFO: ::ffff:127.0.0.1:58526 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2012gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2013gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2014gitlab # [ 417.309820] nixbot[3630]: INFO: ::ffff:127.0.0.1:58532 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2015gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2016gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2017gitlab # [ 418.366223] nixbot[3630]: INFO: ::ffff:127.0.0.1:58536 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2018gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2019gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2020gitlab # [ 419.422817] nixbot[3630]: INFO: ::ffff:127.0.0.1:58552 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2021gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2022gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2023gitlab # [ 420.479807] nixbot[3630]: INFO: ::ffff:127.0.0.1:58566 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2024gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2025gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2026gitlab # [ 421.536440] nixbot[3630]: INFO: ::ffff:127.0.0.1:43202 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2027gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2028gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2029gitlab # [ 422.569515] nixbot[3630]: INFO: ::ffff:127.0.0.1:43204 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2030gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.03 seconds)2031gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2032gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)2033gitlab # [ 423.622827] nixbot[3630]: INFO: ::ffff:127.0.0.1:43210 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2034gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2035gitlab # [ 424.677513] nixbot[3630]: INFO: ::ffff:127.0.0.1:43226 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2036gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2037gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2038gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)2039gitlab # [ 425.729577] nixbot[3630]: INFO: ::ffff:127.0.0.1:43232 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2040gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2041gitlab # [ 426.785549] nixbot[3630]: INFO: ::ffff:127.0.0.1:43240 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2042gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2043gitlab # [ 427.591602] postgres[4154]: [4154] ERROR: relation "table_without_models" does not exist2044gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2045gitlab # [ 427.593538] postgres[4154]: [4154] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')2046gitlab # [ 427.842439] nixbot[3630]: INFO: ::ffff:127.0.0.1:43246 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2047gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2048gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2049gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)2050gitlab # [ 428.904208] nixbot[3630]: INFO: ::ffff:127.0.0.1:43254 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2051gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2052gitlab # [ 429.958699] nixbot[3630]: INFO: ::ffff:127.0.0.1:43260 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2053gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2054gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2055gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)2056gitlab # [ 431.016222] nixbot[3630]: INFO: ::ffff:127.0.0.1:41344 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2057gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2058gitlab # [ 432.071931] nixbot[3630]: INFO: ::ffff:127.0.0.1:41350 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2059gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2060gitlab # [ 432.457524] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:15.350Z","message":"Booted Rails 7.2.3.1 application in production environment"}2061gitlab # [ 432.464071] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:15.357Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}2062gitlab # [ 432.466081] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:15.357Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}2063gitlab # [ 432.467836] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:15.357Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}2064gitlab # [ 432.472069] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:15.365Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}2065gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2066gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)2067gitlab # [ 433.130299] nixbot[3630]: INFO: ::ffff:127.0.0.1:41358 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2068gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2069gitlab # [ 434.185930] nixbot[3630]: INFO: ::ffff:127.0.0.1:41374 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2070gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2071gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2072gitlab # [ 435.274842] nixbot[3630]: INFO: ::ffff:127.0.0.1:41388 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2073gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.09 seconds)2074gitlab # [ 435.282130] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:18.168Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"6f3c358cb7ebca5c36b46e18","created_at":"2026-08-28T11:40:45.390Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:45.439Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-6f3c358cb7ebca5c36b46e18: start","job_status":"start","queue_duration_s":92.728677,"scheduling_latency_s":92.728739}2075gitlab # [ 435.564695] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:18.457Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"6f3c358cb7ebca5c36b46e18","created_at":"2026-08-28T11:40:45.390Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:45.439Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-6f3c358cb7ebca5c36b46e18: done: 0.288893 sec","job_status":"done","queue_duration_s":92.728677,"scheduling_latency_s":92.728739,"redis_calls":23,"redis_duration_s":0.009142,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.004544,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002652,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14893546,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.288893,"completed_at":"2026-08-28T11:42:18.456Z","load_balancing_strategy":"primary","db_duration_s":0.085089,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2076gitlab # [ 435.682790] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:18.460Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"9f0a944de72c40cfd261da89","created_at":"2026-08-28T11:40:45.456Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:45.463Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-9f0a944de72c40cfd261da89: start","job_status":"start","queue_duration_s":92.996702,"scheduling_latency_s":92.99675}2077gitlab # [ 435.766363] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:18.659Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"9f0a944de72c40cfd261da89","created_at":"2026-08-28T11:40:45.456Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:45.463Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-9f0a944de72c40cfd261da89: done: 0.199042 sec","job_status":"done","queue_duration_s":92.996702,"scheduling_latency_s":92.99675,"redis_calls":14,"redis_duration_s":0.006332,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001576,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002897,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001859,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.145919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15136318,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.199042,"completed_at":"2026-08-28T11:42:18.659Z","load_balancing_strategy":"primary","db_duration_s":0.04112,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2078gitlab # [ 435.858222] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:18.662Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"a0af21246e6775bacd3c00aa","created_at":"2026-08-28T11:40:46.177Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:46.185Z","job_size_bytes":5,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-a0af21246e6775bacd3c00aa: start","job_status":"start","queue_duration_s":92.476389,"scheduling_latency_s":92.47642}2079gitlab # [ 435.980556] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:18.873Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"a0af21246e6775bacd3c00aa","created_at":"2026-08-28T11:40:46.177Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:46.185Z","job_size_bytes":5,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-a0af21246e6775bacd3c00aa: done: 0.211672 sec","job_status":"done","queue_duration_s":92.476389,"scheduling_latency_s":92.47642,"redis_calls":12,"redis_duration_s":0.003525,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00091,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002064,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.159664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035498,"duration_s":0.211672,"completed_at":"2026-08-28T11:42:18.873Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04441,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2080gitlab # [ 436.075871] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:18.875Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"42d0ab66967e18d0436f1a19","created_at":"2026-08-28T11:40:46.224Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:46.231Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-42d0ab66967e18d0436f1a19: start","job_status":"start","queue_duration_s":92.643834,"scheduling_latency_s":92.643856}2081gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2082gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.07 seconds)2083gitlab # [ 436.256964] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:19.150Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"42d0ab66967e18d0436f1a19","created_at":"2026-08-28T11:40:46.224Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:46.231Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-42d0ab66967e18d0436f1a19: done: 0.274353 sec","job_status":"done","queue_duration_s":92.643834,"scheduling_latency_s":92.643856,"redis_calls":11,"redis_duration_s":0.003537,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000825,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.231,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.231,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.164925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00076062,"duration_s":0.274353,"completed_at":"2026-08-28T11:42:19.150Z","load_balancing_strategy":"primary","db_duration_s":0.112105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084gitlab # [ 436.369182] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:19.152Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"2d30d96e4b975f9062ebbcb5","created_at":"2026-08-28T11:40:46.236Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:46.242Z","job_size_bytes":89,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-2d30d96e4b975f9062ebbcb5: start","job_status":"start","queue_duration_s":92.909984,"scheduling_latency_s":92.910008}2085gitlab # [ 436.397233] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:19.208Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"2d30d96e4b975f9062ebbcb5","created_at":"2026-08-28T11:40:46.236Z","trace_propagation_headers":{"sentry-trace":"5f41933f36c843e3828b909a1806bfcc-54f5c4f6eff84864","baggage":"sentry-trace_id=5f41933f36c843e3828b909a1806bfcc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R5C6Q34JMSEZ5FGZ6YMX","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:46.242Z","job_size_bytes":89,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-2d30d96e4b975f9062ebbcb5: done: 0.05587 sec","job_status":"done","queue_duration_s":92.909984,"scheduling_latency_s":92.910008,"redis_calls":11,"redis_duration_s":0.0048059999999999995,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000867,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00105518,"duration_s":0.05587,"completed_at":"2026-08-28T11:42:19.208Z","load_balancing_strategy":"primary","db_duration_s":0.010145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2086gitlab # [ 436.449405] nixbot[3630]: INFO: ::ffff:127.0.0.1:41392 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2087gitlab # [ 436.451673] gitaly[907]: time="2026-08-28T11:42:19.310Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:29.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:19.294" grpc.time_ms=15.512 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088gitlab # [ 436.465301] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:19.210Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 39ac7f2789de299302f12fa1c23545460e749303 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"109e487a248a7186671a6b0f","created_at":"2026-08-28T11:40:49.528Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-59494b3287a54eef","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:49.574Z","job_size_bytes":185,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-109e487a248a7186671a6b0f: start","job_status":"start","queue_duration_s":89.636193,"scheduling_latency_s":89.636214}2089gitlab # [ 436.482183] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:19.233Z","message":"changes: \"0000000000000000000000000000000000000000 39ac7f2789de299302f12fa1c23545460e749303 refs/heads/master\\n\""}2090gitlab # [ 436.484659] gitaly[907]: time="2026-08-28T11:42:19.337Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-28T11:42:29.030" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:19.321" grpc.time_ms=16.344 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091gitlab # [ 436.558899] gitaly[907]: time="2026-08-28T11:42:19.452Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-28T11:42:29.144" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:19.434" grpc.time_ms=17.795 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2092gitlab # [ 436.636281] gitaly[907]: time="2026-08-28T11:42:19.529Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:19.528" grpc.time_ms=1.337 pid=907 span.kind=server system=grpc2093gitlab # [ 436.709937] gitaly[907]: time="2026-08-28T11:42:19.603Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:19.602" grpc.time_ms=0.992 pid=907 span.kind=server system=grpc2094gitlab # [ 436.737951] gitaly[907]: time="2026-08-28T11:42:19.618Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 diskcache=ef98a0e4-f271-482e-96c2-ce7426be78b6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-28T11:42:29.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:19.472" pid=907 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2095gitlab # [ 436.759494] gitaly[907]: time="2026-08-28T11:42:19.618Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=138 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5216 command.oublock=8 command.real_time_ms=138 command.spawn_token_fork_ms=1 command.system_time_ms=74 command.user_time_ms=64 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-28T11:42:29.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:19.472" grpc.time_ms=145.801 pid=907 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2096gitlab # [ 436.780724] gitaly[907]: time="2026-08-28T11:42:19.651Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-28T11:42:29.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:19.650" grpc.time_ms=0.428 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097gitlab # [ 436.794357] gitaly[907]: time="2026-08-28T11:42:19.685Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=20 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-28T11:42:29.018" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:19.659" grpc.time_ms=26.078 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098gitlab # [ 436.981455] gitaly[907]: time="2026-08-28T11:42:19.875Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=314 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-08-28T11:42:49.061" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:19.862" grpc.time_ms=12.7 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2099gitlab # [ 437.045321] gitaly[907]: time="2026-08-28T11:42:19.938Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-28T11:42:29.007" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:19.938" grpc.time_ms=0.652 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab # [ 437.076910] gitaly[907]: time="2026-08-28T11:42:19.955Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-28T11:42:29.013" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-08-28T11:42:19.943" grpc.time_ms=11.904 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2101gitlab # [ 437.110908] gitaly[907]: time="2026-08-28T11:42:20.004Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:29.055" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:19.985" grpc.time_ms=18.538 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102gitlab # [ 437.141922] gitaly[907]: time="2026-08-28T11:42:20.035Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:30.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:20.017" grpc.time_ms=18.033 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2103gitlab # [ 437.194392] gitaly[907]: time="2026-08-28T11:42:20.088Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-28T11:42:50.058" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:20.058" grpc.time_ms=28.96 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2105gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.07 seconds)2106gitlab # [ 437.414198] nixbot[3630]: INFO: ::ffff:127.0.0.1:41406 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2107gitlab # [ 437.923338] gitaly[907]: time="2026-08-28T11:42:20.816Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-28T11:42:30.173" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:20.814" grpc.time_ms=2.246 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2108gitlab # [ 438.005348] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:20.896Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 39ac7f2789de299302f12fa1c23545460e749303 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"109e487a248a7186671a6b0f","created_at":"2026-08-28T11:40:49.528Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-59494b3287a54eef","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:49.574Z","job_size_bytes":185,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-109e487a248a7186671a6b0f: done: 1.68599 sec","job_status":"done","queue_duration_s":89.636193,"scheduling_latency_s":89.636214,"gitaly_calls":13,"gitaly_duration_s":0.383632,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.079808,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.007781,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.011453,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.003392,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005943,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.03185,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.019389,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.096,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.314,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.366,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.096,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.146268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00167352,"exclusive_lock_hold_duration_s":0.05464262,"duration_s":1.68599,"completed_at":"2026-08-28T11:42:20.896Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.168146,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2109gitlab # [ 438.107316] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:20.900Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","39ac7f2789de299302f12fa1c23545460e749303 f1bc3ee186b853356efea66d316b0d35f7e7dd26 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"9de8a81abc3d6ca0f3fead5d","created_at":"2026-08-28T11:40:54.415Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-bd004f11a052488e","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:54.419Z","job_size_bytes":185,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-9de8a81abc3d6ca0f3fead5d: start","job_status":"start","queue_duration_s":86.480248,"scheduling_latency_s":86.480295}2110gitlab # [ 438.126634] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:20.924Z","message":"changes: \"39ac7f2789de299302f12fa1c23545460e749303 f1bc3ee186b853356efea66d316b0d35f7e7dd26 refs/heads/master\\n\""}2111gitlab # [ 438.129226] gitaly[907]: time="2026-08-28T11:42:20.976Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:30.027" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:20.957" grpc.time_ms=18.456 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab # [ 438.142651] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:21.013Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ceef65676c8f2d401c2c9595","created_at":"2026-08-28T11:42:21.009Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-37d15465bbfd47a7","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"0cd7ed81e7890a4edcb3c141","job_size_bytes":46,"pid":3572,"job_status":"deduplicated","message":"ProjectCacheWorker JID-ceef65676c8f2d401c2c9595: deduplicated: until executing","deduplication.type":"until executing"}2113gitlab # [ 438.158877] gitaly[907]: time="2026-08-28T11:42:21.004Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-28T11:42:30.056" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:20.987" grpc.time_ms=17.101 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2114gitlab # [ 438.172528] gitaly[907]: time="2026-08-28T11:42:21.058Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-28T11:42:31.028" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:21.039" grpc.time_ms=18.95 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2115gitlab # [ 438.191518] gitaly[907]: time="2026-08-28T11:42:21.085Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-08-28T11:42:51.065" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:21.066" grpc.time_ms=18.327 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 438.224643] gitaly[907]: time="2026-08-28T11:42:21.118Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-28T11:42:31.097" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-08-28T11:42:21.107" grpc.time_ms=10.657 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2117gitlab # [ 438.260634] gitaly[907]: time="2026-08-28T11:42:21.154Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:31.126" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:21.137" grpc.time_ms=16.866 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2118gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2119gitlab # [ 438.460901] nixbot[3630]: INFO: ::ffff:127.0.0.1:41422 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2120gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.05 seconds)2121gitlab # [ 438.468517] gitaly[907]: time="2026-08-28T11:42:21.362Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-28T11:42:31.005" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:21.357" grpc.time_ms=4.863 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2122gitlab # [ 438.518601] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:21.411Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","39ac7f2789de299302f12fa1c23545460e749303 f1bc3ee186b853356efea66d316b0d35f7e7dd26 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"9de8a81abc3d6ca0f3fead5d","created_at":"2026-08-28T11:40:54.415Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-bd004f11a052488e","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:40:54.419Z","job_size_bytes":185,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-9de8a81abc3d6ca0f3fead5d: done: 0.511034 sec","job_status":"done","queue_duration_s":86.480248,"scheduling_latency_s":86.480295,"gitaly_calls":7,"gitaly_duration_s":0.13703,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.04031,"redis_read_bytes":500,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.005311,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001626,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006671,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.012144,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.014558,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.045,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.045,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.324027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00126268,"exclusive_lock_hold_duration_s":0.03570232,"duration_s":0.511034,"completed_at":"2026-08-28T11:42:21.411Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051017,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2123gitlab # [ 438.574435] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:21.414Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"913ac8b9ebd548988ef6e98d","created_at":"2026-08-28T11:40:55.701Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-4029c28a0fcb455e","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:55.711Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-913ac8b9ebd548988ef6e98d: start","job_status":"start","queue_duration_s":85.702869,"scheduling_latency_s":85.702914}2124gitlab # [ 438.672666] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:21.565Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"913ac8b9ebd548988ef6e98d","created_at":"2026-08-28T11:40:55.701Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-4029c28a0fcb455e","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:55.711Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-913ac8b9ebd548988ef6e98d: done: 0.151041 sec","job_status":"done","queue_duration_s":85.702869,"scheduling_latency_s":85.702914,"redis_calls":16,"redis_duration_s":0.006156,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00298,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002429,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000747,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068984,"duration_s":0.151041,"completed_at":"2026-08-28T11:42:21.565Z","load_balancing_strategy":"primary","db_duration_s":0.035996,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2125gitlab # [ 438.759923] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:21.568Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9061b7fbc84fd83372ffb152","created_at":"2026-08-28T11:40:55.712Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-4029c28a0fcb455e","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:55.720Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9061b7fbc84fd83372ffb152: start","job_status":"start","queue_duration_s":85.847957,"scheduling_latency_s":85.84799}2126gitlab # [ 439.011948] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:21.903Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9061b7fbc84fd83372ffb152","created_at":"2026-08-28T11:40:55.712Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-4029c28a0fcb455e","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:55.720Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9061b7fbc84fd83372ffb152: done: 0.335499 sec","job_status":"done","queue_duration_s":85.847957,"scheduling_latency_s":85.84799,"redis_calls":21,"redis_duration_s":0.008291,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003277,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002766,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.263884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00094512,"duration_s":0.335499,"completed_at":"2026-08-28T11:42:21.903Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2127gitlab # [ 439.074384] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:21.907Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f8c39f3eeb0912b9883c56aa","created_at":"2026-08-28T11:40:55.891Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-4029c28a0fcb455e","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:55.900Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f8c39f3eeb0912b9883c56aa: start","job_status":"start","queue_duration_s":86.007028,"scheduling_latency_s":86.007071}2128gitlab # [ 439.390143] gitaly[907]: time="2026-08-28T11:42:22.282Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:22.280" grpc.time_ms=1.226 pid=907 span.kind=server system=grpc2129gitlab # [ 439.435314] gitaly[907]: time="2026-08-28T11:42:22.328Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:22.327" grpc.time_ms=0.734 pid=907 span.kind=server system=grpc2130gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2131gitlab # [ 439.498386] gitaly[907]: time="2026-08-28T11:42:22.392Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:22.390" grpc.time_ms=1.283 pid=907 span.kind=server system=grpc2132gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.07 seconds)2133gitlab # [ 439.528204] gitaly[907]: time="2026-08-28T11:42:22.406Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H diskcache=1a2ba62a-f57c-4f42-8ad8-ebdfc1df456f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:42:52.220" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:22.221" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134gitlab # [ 439.549698] gitaly[907]: time="2026-08-28T11:42:22.407Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=199 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7331 command.oublock=0 command.real_time_ms=176 command.spawn_token_fork_ms=1 command.system_time_ms=105 command.user_time_ms=94 component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:42:52.220" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:22.221" grpc.time_ms=185.792 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2135gitlab # [ 439.576215] nixbot[3630]: INFO: ::ffff:127.0.0.1:41434 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2136gitlab # [ 439.799361] gitaly[907]: time="2026-08-28T11:42:22.692Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-28T11:42:52.690" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:22.690" grpc.time_ms=1.752 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2137gitlab # [ 439.866230] gitaly[907]: time="2026-08-28T11:42:22.759Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RGF9D7SRF7HY0BSAQP1H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:52.757" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:22.757" grpc.time_ms=1.581 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2138gitlab # [ 439.916395] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:22.809Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f8c39f3eeb0912b9883c56aa","created_at":"2026-08-28T11:40:55.891Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-4029c28a0fcb455e","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-28T11:40:55.900Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f8c39f3eeb0912b9883c56aa: done: 0.901245 sec","job_status":"done","queue_duration_s":86.007028,"scheduling_latency_s":86.007071,"gitaly_calls":3,"gitaly_duration_s":0.201744,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.056192,"redis_read_bytes":457,"redis_write_bytes":27999,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002462,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002083,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.022429,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.008161,"redis_queues_read_bytes":36,"redis_queues_write_bytes":14493,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.012291,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001516,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000915,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003873,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.112,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.052,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.058,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.056,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.547601,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.7624398,"duration_s":0.901245,"completed_at":"2026-08-28T11:42:22.808Z","load_balancing_strategy":"primary","db_duration_s":0.133248,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2139gitlab # [ 439.971959] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:22.812Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0cd7ed81e7890a4edcb3c141","created_at":"2026-08-28T11:42:19.343Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:19.379Z","job_size_bytes":46,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0cd7ed81e7890a4edcb3c141: start","job_status":"start","queue_duration_s":3.433119,"scheduling_latency_s":3.433162}2140gitlab # [ 440.008319] gitaly[907]: time="2026-08-28T11:42:22.901Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-08-28T17:42:22.889" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:22.890" grpc.time_ms=11.028 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2141gitlab # [ 440.037078] gitaly[907]: time="2026-08-28T11:42:22.930Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-28T11:42:52.012" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:22.913" grpc.time_ms=16.667 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2142gitlab # [ 440.105019] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:22.996Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0cd7ed81e7890a4edcb3c141","created_at":"2026-08-28T11:42:19.343Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:19.379Z","job_size_bytes":46,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0cd7ed81e7890a4edcb3c141: done: 0.184636 sec","job_status":"done","queue_duration_s":3.433119,"scheduling_latency_s":3.433162,"gitaly_calls":2,"gitaly_duration_s":0.035595,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011422,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002683,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.001108,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00261,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004067,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00078172,"duration_s":0.184636,"completed_at":"2026-08-28T11:42:22.996Z","load_balancing_strategy":"primary","db_duration_s":0.021803,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2143gitlab # [ 440.184871] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:23.000Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f5a9226342ee65eae08c9987","created_at":"2026-08-28T11:42:19.413Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:19.425Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f5a9226342ee65eae08c9987: start","job_status":"start","queue_duration_s":3.575499,"scheduling_latency_s":3.575534}2144gitlab # [ 440.204883] gitaly[907]: time="2026-08-28T11:42:23.095Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 diskcache=9592e42f-094f-4a61-b1c1-1aac8eeea902 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-28T17:42:23.053" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:23.053" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2145gitlab # [ 440.215492] gitaly[907]: time="2026-08-28T11:42:23.096Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=49 command.spawn_token_fork_ms=4 command.system_time_ms=28 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-28T17:42:23.053" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:23.053" grpc.time_ms=42.324 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2146gitlab # [ 440.293960] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:23.186Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f5a9226342ee65eae08c9987","created_at":"2026-08-28T11:42:19.413Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:19.425Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f5a9226342ee65eae08c9987: done: 0.185959 sec","job_status":"done","queue_duration_s":3.575499,"scheduling_latency_s":3.575534,"gitaly_calls":1,"gitaly_duration_s":0.047015,"redis_calls":15,"redis_duration_s":0.006814,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000847,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001342,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002192,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.045,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13038448,"duration_s":0.185959,"completed_at":"2026-08-28T11:42:23.186Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028622,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2147gitlab # [ 440.370265] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:23.190Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0108fdf9d48177df0bb2f3ab","created_at":"2026-08-28T11:42:19.641Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:19.645Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0108fdf9d48177df0bb2f3ab: start","job_status":"start","queue_duration_s":3.544943,"scheduling_latency_s":3.544991}2148gitlab # [ 440.387144] gitaly[907]: time="2026-08-28T11:42:23.280Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-28T11:42:53.262" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:23.262" grpc.time_ms=17.726 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2149gitlab # [ 440.412292] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:23.304Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0108fdf9d48177df0bb2f3ab","created_at":"2026-08-28T11:42:19.641Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:19.645Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0108fdf9d48177df0bb2f3ab: done: 0.114431 sec","job_status":"done","queue_duration_s":3.544943,"scheduling_latency_s":3.544991,"gitaly_calls":1,"gitaly_duration_s":0.022142,"redis_calls":16,"redis_duration_s":0.008461,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000549,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.003624,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001733,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.114431,"completed_at":"2026-08-28T11:42:23.304Z","load_balancing_strategy":"primary","db_duration_s":0.006258,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2150gitlab # [ 440.469135] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:23.308Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2a67da7ccb50e840644a7b43","created_at":"2026-08-28T11:42:20.523Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:373d96bdd0580b63cb830051fc349e857a8e35875ddfe433a2fd28e6049e94fc","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:20.537Z","job_size_bytes":467,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2a67da7ccb50e840644a7b43: start","job_status":"start","queue_duration_s":2.771027,"scheduling_latency_s":2.771069}2151gitlab # [ 440.487376] gitaly[907]: time="2026-08-28T11:42:23.364Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:53.362" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:23.363" grpc.time_ms=1.232 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2152gitlab # [ 440.500000] gitaly[907]: time="2026-08-28T11:42:23.387Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-28T11:42:33.019" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:23.370" grpc.time_ms=16.972 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2153gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2154gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.07 seconds)2155gitlab # [ 440.606831] nixbot[3630]: INFO: ::ffff:127.0.0.1:41450 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2156gitlab # [ 440.739606] gitaly[907]: time="2026-08-28T11:42:23.633Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-28T11:42:53.631" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:23.631" grpc.time_ms=1.329 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2157gitlab # [ 440.759068] gitaly[907]: time="2026-08-28T11:42:23.643Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=88 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-28T11:42:53.638" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:23.639" grpc.time_ms=4.266 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158gitlab # [ 440.903520] gitaly[907]: time="2026-08-28T11:42:23.797Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:33.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:23.777" grpc.time_ms=19.142 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2159gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2160gitlab # [ 441.664579] nixbot[3630]: INFO: ::ffff:127.0.0.1:38726 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2161gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2162gitlab # [ 441.765341] gitaly[907]: time="2026-08-28T11:42:24.657Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=2 command.system_time_ms=22 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-28T11:42:54.628" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:24.628" grpc.time_ms=28.936 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2163gitlab # [ 441.807661] gitaly[907]: time="2026-08-28T11:42:24.682Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142R9WA3FDZ5899KKJJ6494 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-28T11:42:54.663" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:24.664" grpc.time_ms=18.133 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2164gitlab # [ 441.838537] sidekiq-cluster[3572]: {"severity":"WARN","time":"2026-08-28T11:42:24.731Z","class":"CreatePipelineWorker","correlation_id":"01M142R9WA3FDZ5899KKJJ6494","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"39ac7f2789de299302f12fa1c23545460e749303","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"39ac7f2789de299302f12fa1c23545460e749303","ref":"refs/heads/master"},"retry":0}2165gitlab # [ 441.852058] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:24.743Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2a67da7ccb50e840644a7b43","created_at":"2026-08-28T11:42:20.523Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:373d96bdd0580b63cb830051fc349e857a8e35875ddfe433a2fd28e6049e94fc","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:20.537Z","job_size_bytes":467,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2a67da7ccb50e840644a7b43: done: 1.435041 sec","job_status":"done","queue_duration_s":2.771027,"scheduling_latency_s":2.771069,"gitaly_calls":7,"gitaly_duration_s":0.11745,"redis_calls":51,"redis_duration_s":0.017283,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.001616,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.007576,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004526,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001501,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.233044,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048844,"duration_s":1.435041,"completed_at":"2026-08-28T11:42:24.743Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.092056,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2166gitlab # [ 441.917258] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:24.747Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"9dbf601a-9444-4f50-9b22-d048effb9421\"}"],"class":"WebHookWorker","jid":"5a9c678759440f7d5a7d39cc","created_at":"2026-08-28T11:42:20.770Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:edf029ade486658c603cf0f1cf6ebd6f8af1ee5fb78fa454750b6c164321954d","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:20.777Z","job_size_bytes":1751,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5a9c678759440f7d5a7d39cc: start","job_status":"start","queue_duration_s":3.969125,"scheduling_latency_s":3.96916}2167gitlab # [ 442.038679] nixbot[3630]: INFO: ::ffff:127.0.0.1:38730 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2168gitlab # [ 442.081872] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:24.974Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"9dbf601a-9444-4f50-9b22-d048effb9421\"}"],"class":"WebHookWorker","jid":"5a9c678759440f7d5a7d39cc","created_at":"2026-08-28T11:42:20.770Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-5c8b3cac157147aa","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:edf029ade486658c603cf0f1cf6ebd6f8af1ee5fb78fa454750b6c164321954d","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:20.777Z","job_size_bytes":1751,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5a9c678759440f7d5a7d39cc: done: 0.227485 sec","job_status":"done","queue_duration_s":3.969125,"scheduling_latency_s":3.96916,"redis_calls":19,"redis_duration_s":0.006894,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001621,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000875,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002786,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001612,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07597019999997201,"cpu_s":0.129267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00078304,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.227485,"completed_at":"2026-08-28T11:42:24.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005313,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2169gitlab # [ 442.189875] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:24.978Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"49a53d646ce477b307fcc877","created_at":"2026-08-28T11:42:21.026Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-37d15465bbfd47a7","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"f5a9226342ee65eae08c9987","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:21.030Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-49a53d646ce477b307fcc877: start","job_status":"start","queue_duration_s":3.946949,"scheduling_latency_s":3.947027}2170gitlab # [ 442.212987] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:25.079Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"49a53d646ce477b307fcc877","created_at":"2026-08-28T11:42:21.026Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-37d15465bbfd47a7","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"f5a9226342ee65eae08c9987","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:21.030Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-49a53d646ce477b307fcc877: done: 0.101539 sec","job_status":"done","queue_duration_s":3.946949,"scheduling_latency_s":3.947027,"gitaly_calls":1,"gitaly_duration_s":0.020171,"redis_calls":12,"redis_duration_s":0.006678,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001755,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0547748,"duration_s":0.101539,"completed_at":"2026-08-28T11:42:25.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2171gitlab # [ 442.262496] gitaly[907]: time="2026-08-28T11:42:25.040Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G diskcache=07332558-21a2-4edb-a813-4acf7017439d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-28T17:42:25.024" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.025" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2172gitlab # [ 442.272580] gitaly[907]: time="2026-08-28T11:42:25.041Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-28T17:42:25.024" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.025" grpc.time_ms=15.302 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2173gitlab # [ 442.285764] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:25.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"93beb0c271346f03c23502c1","created_at":"2026-08-28T11:42:21.266Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-37d15465bbfd47a7","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:74812de2a72db536e0b79ada7ec8c2b139d2dec615242a0218e1712f639bc1a3","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:21.272Z","job_size_bytes":467,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-93beb0c271346f03c23502c1: start","job_status":"start","queue_duration_s":3.810143,"scheduling_latency_s":3.810178}2174gitlab # [ 442.303090] gitaly[907]: time="2026-08-28T11:42:25.125Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:55.123" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.124" grpc.time_ms=1.079 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175gitlab # [ 442.315534] gitaly[907]: time="2026-08-28T11:42:25.144Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-28T11:42:35.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:25.130" grpc.time_ms=14.136 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2176gitlab # [ 442.328728] gitaly[907]: time="2026-08-28T11:42:25.206Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-28T11:42:55.003" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.204" grpc.time_ms=2.345 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2177gitlab # [ 442.341319] gitaly[907]: time="2026-08-28T11:42:25.218Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=22 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-28T11:42:55.014" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.214" grpc.time_ms=3.665 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2178gitlab # [ 442.370764] gitaly[907]: time="2026-08-28T11:42:25.264Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:35.130" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:25.250" grpc.time_ms=13.424 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2179gitlab # [ 442.402792] gitaly[907]: time="2026-08-28T11:42:25.296Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142V86P7BJHHGG78KZ94AS5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:25.286" grpc.time_ms=10.285 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2180gitlab # [ 442.417878] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 3142181gitlab # [ 442.583395] gitaly[907]: time="2026-08-28T11:42:25.476Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142V8GN4YEKZVMGR57JM9N6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:25.456" pid=907 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2182gitlab # [ 442.615776] gitaly[907]: time="2026-08-28T11:42:25.477Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V8GN4YEKZVMGR57JM9N6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:25.456" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2183gitlab # [ 442.643600] gitlab-workhorse[820]: time="2026-08-28T11:42:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672184gitlab # [ 442.646806] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1772185gitlab # [ 442.649985] nixbot[3630]: {"ts": "2026-08-28T11:42:25.534Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2186gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2187gitlab # [ 442.655095] gitaly[907]: time="2026-08-28T11:42:25.477Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V8GN4YEKZVMGR57JM9N6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:25.456" grpc.time_ms=21.009 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2188gitlab # [ 442.736237] nixbot[3630]: INFO: ::ffff:127.0.0.1:38734 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2189gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.07 seconds)2190gitlab # [ 442.795947] gitaly[907]: time="2026-08-28T11:42:25.688Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:55.686" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.686" grpc.time_ms=1.376 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2191gitlab # [ 442.911255] gitaly[907]: time="2026-08-28T11:42:25.804Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:35.275" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:25.786" grpc.time_ms=18.195 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2192gitlab # [ 443.042174] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3742193gitlab # [ 443.047530] nixbot[3630]: {"ts": "2026-08-28T11:42:25.938Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 \"HTTP/1.1 201 Created\""}2194gitlab # [ 443.072335] gitaly[907]: time="2026-08-28T11:42:25.965Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=5 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-28T11:42:55.740" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.940" grpc.time_ms=24.871 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2195gitlab # [ 443.110278] gitaly[907]: time="2026-08-28T11:42:25.991Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142RF3WDRRZ0V4BT8HPXT7G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-28T11:42:55.769" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:25.970" grpc.time_ms=21.617 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2196gitlab # [ 443.132676] sidekiq-cluster[3572]: {"severity":"WARN","time":"2026-08-28T11:42:26.019Z","class":"CreatePipelineWorker","correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"39ac7f2789de299302f12fa1c23545460e749303","after":"f1bc3ee186b853356efea66d316b0d35f7e7dd26","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"f1bc3ee186b853356efea66d316b0d35f7e7dd26","ref":"refs/heads/master"},"retry":0}2197gitlab # [ 443.142482] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.027Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"93beb0c271346f03c23502c1","created_at":"2026-08-28T11:42:21.266Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-37d15465bbfd47a7","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:74812de2a72db536e0b79ada7ec8c2b139d2dec615242a0218e1712f639bc1a3","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:21.272Z","job_size_bytes":467,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-93beb0c271346f03c23502c1: done: 0.944855 sec","job_status":"done","queue_duration_s":3.810143,"scheduling_latency_s":3.810178,"gitaly_calls":7,"gitaly_duration_s":0.109746,"redis_calls":33,"redis_duration_s":0.0065060000000000005,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000863,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001479,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003053,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.805761,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.944855,"completed_at":"2026-08-28T11:42:26.027Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0317,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2198gitlab # [ 443.195035] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.029Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"277aef2f-9e0d-432b-8969-560862e4da42\"}"],"class":"WebHookWorker","jid":"5354c510e30fd2259dbe44b8","created_at":"2026-08-28T11:42:21.309Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-37d15465bbfd47a7","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0b0296f72f55ddbf534d6e0642b055746ffbf28a7ee1a40364f5a1cf0e7e095a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:21.315Z","job_size_bytes":1749,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5354c510e30fd2259dbe44b8: start","job_status":"start","queue_duration_s":4.714304,"scheduling_latency_s":4.714338}2199gitlab # [ 443.212819] gitaly[907]: time="2026-08-28T11:42:26.029Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V94VF4EGG8T24T3Q93PQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:56.027" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:26.028" grpc.time_ms=1.181 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200gitlab # [ 443.225675] nixbot[3630]: INFO: ::ffff:127.0.0.1:38748 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2201gitlab # [ 443.226936] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.099Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"277aef2f-9e0d-432b-8969-560862e4da42\"}"],"class":"WebHookWorker","jid":"5354c510e30fd2259dbe44b8","created_at":"2026-08-28T11:42:21.309Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-37d15465bbfd47a7","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0b0296f72f55ddbf534d6e0642b055746ffbf28a7ee1a40364f5a1cf0e7e095a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:21.315Z","job_size_bytes":1749,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5354c510e30fd2259dbe44b8: done: 0.069912 sec","job_status":"done","queue_duration_s":4.714304,"scheduling_latency_s":4.714338,"redis_calls":14,"redis_duration_s":0.005622,"redis_read_bytes":26,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.001495,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00301,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.01659604000002446,"cpu_s":0.038765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.069912,"completed_at":"2026-08-28T11:42:26.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 443.276001] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.104Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2f33e871ee52b610d2d2616b","created_at":"2026-08-28T11:42:22.179Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.194Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2f33e871ee52b610d2d2616b: start","job_status":"start","queue_duration_s":3.909727,"scheduling_latency_s":3.909772}2203gitlab # [ 443.292501] gitaly[907]: time="2026-08-28T11:42:26.109Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142V94VF4EGG8T24T3Q93PQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:36.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:26.099" grpc.time_ms=9.163 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2204gitlab # [ 443.305896] nixbot[3630]: {"ts": "2026-08-28T11:42:26.177Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 \"HTTP/1.1 201 Created\""}2205gitlab # [ 443.308661] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2282206gitlab # [ 443.338273] gitaly[907]: time="2026-08-28T11:42:26.232Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M142V9ANQP6YWVQ2RQM0V55J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.223" grpc.time_ms=8.092 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2207gitlab # [ 443.351938] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1002208gitlab # [ 443.356726] gitaly[907]: time="2026-08-28T11:42:26.250Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9C7BS1REGPSBXZ5R7W6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:56.246" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:26.247" grpc.time_ms=3.376 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2209gitlab # [ 443.386057] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.279Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2f33e871ee52b610d2d2616b","created_at":"2026-08-28T11:42:22.179Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.194Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2f33e871ee52b610d2d2616b: done: 0.175044 sec","job_status":"done","queue_duration_s":3.909727,"scheduling_latency_s":3.909772,"redis_calls":10,"redis_duration_s":0.018062,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.009658,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060464,"duration_s":0.175044,"completed_at":"2026-08-28T11:42:26.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025856,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2210gitlab # [ 443.435934] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.284Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cf5d1efd3939fe81b4722b3c","created_at":"2026-08-28T11:42:22.417Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.432Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cf5d1efd3939fe81b4722b3c: start","job_status":"start","queue_duration_s":3.852765,"scheduling_latency_s":3.852795}2211gitlab # [ 443.453792] gitlab-workhorse[820]: time="2026-08-28T11:42:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672212gitlab # [ 443.459997] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 732213gitlab # [ 443.461943] gitaly[907]: time="2026-08-28T11:42:26.302Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142V9C7BS1REGPSBXZ5R7W6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:36.161" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:26.291" grpc.time_ms=10.56 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2214gitlab # [ 443.477074] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.341Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cf5d1efd3939fe81b4722b3c","created_at":"2026-08-28T11:42:22.417Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.432Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cf5d1efd3939fe81b4722b3c: done: 0.056979 sec","job_status":"done","queue_duration_s":3.852765,"scheduling_latency_s":3.852795,"redis_calls":10,"redis_duration_s":0.007797999999999999,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000674,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006036,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004994,"duration_s":0.056979,"completed_at":"2026-08-28T11:42:26.341Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2215gitlab # [ 443.522384] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.346Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7eed30fbaf5dd50cb9691fee","created_at":"2026-08-28T11:42:22.458Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.476Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7eed30fbaf5dd50cb9691fee: start","job_status":"start","queue_duration_s":3.86991,"scheduling_latency_s":3.869935}2216gitlab # [ 443.538231] gitaly[907]: time="2026-08-28T11:42:26.308Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142V9DV2WSHRBG8QYT5T1SW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.296" pid=907 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2217gitlab # [ 443.548457] nixbot[3630]: {"ts": "2026-08-28T11:42:26.402Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 \"HTTP/1.1 201 Created\""}2218gitlab # [ 443.551195] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.392Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7eed30fbaf5dd50cb9691fee","created_at":"2026-08-28T11:42:22.458Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.476Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7eed30fbaf5dd50cb9691fee: done: 0.046525 sec","job_status":"done","queue_duration_s":3.86991,"scheduling_latency_s":3.869935,"redis_calls":10,"redis_duration_s":0.003999,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086762,"duration_s":0.046525,"completed_at":"2026-08-28T11:42:26.392Z","load_balancing_strategy":"primary","db_duration_s":0.005266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2219gitlab # [ 443.595859] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.401Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2df707e8d5aff858f0fe611e","created_at":"2026-08-28T11:42:22.507Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.523Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2df707e8d5aff858f0fe611e: start","job_status":"start","queue_duration_s":3.877586,"scheduling_latency_s":3.877616}2220gitlab # [ 443.612147] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/39ac7f2789de299302f12fa1c23545460e749303 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2172221gitlab # [ 443.614617] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 882222gitlab # [ 443.616683] gitaly[907]: time="2026-08-28T11:42:26.308Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9DV2WSHRBG8QYT5T1SW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.296" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2223gitlab # [ 443.629900] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.453Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2df707e8d5aff858f0fe611e","created_at":"2026-08-28T11:42:22.507Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.523Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2df707e8d5aff858f0fe611e: done: 0.05211 sec","job_status":"done","queue_duration_s":3.877586,"scheduling_latency_s":3.877616,"redis_calls":13,"redis_duration_s":0.006816,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002553,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00057,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031132,"duration_s":0.05211,"completed_at":"2026-08-28T11:42:26.453Z","load_balancing_strategy":"primary","db_duration_s":0.005513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 443.675877] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.454Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"660209a37e6cbbb1b564f44f","created_at":"2026-08-28T11:42:22.525Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.540Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-660209a37e6cbbb1b564f44f: start","job_status":"start","queue_duration_s":3.914274,"scheduling_latency_s":3.91429}2225gitlab # [ 443.692969] gitlab-workhorse[820]: time="2026-08-28T11:42:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672226gitlab # [ 443.695906] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 712227gitlab # [ 443.697976] gitaly[907]: time="2026-08-28T11:42:26.308Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9DV2WSHRBG8QYT5T1SW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.296" grpc.time_ms=11.938 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2228gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2229gitlab # [ 443.712898] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.495Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"660209a37e6cbbb1b564f44f","created_at":"2026-08-28T11:42:22.525Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.540Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-660209a37e6cbbb1b564f44f: done: 0.040922 sec","job_status":"done","queue_duration_s":3.914274,"scheduling_latency_s":3.91429,"redis_calls":10,"redis_duration_s":0.001492,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026614,"duration_s":0.040922,"completed_at":"2026-08-28T11:42:26.495Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2230gitlab # [ 443.756803] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.499Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"cc88ffcb3fab03f3ffdd6170","created_at":"2026-08-28T11:42:22.542Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.559Z","job_size_bytes":19,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-cc88ffcb3fab03f3ffdd6170: start","job_status":"start","queue_duration_s":3.939192,"scheduling_latency_s":3.939209}2231gitlab # [ 443.772104] gitaly[907]: time="2026-08-28T11:42:26.447Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=6 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142V9HQWV0A7W5FSMF76BTM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.432" grpc.time_ms=14.576 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2232gitlab # [ 443.785385] gitaly[907]: time="2026-08-28T11:42:26.519Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142V9MHSHX1XZTFT5GXFGTM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.510" pid=907 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2233gitlab # [ 443.795839] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 792234gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.06 seconds)2235gitlab # [ 443.798132] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.540Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"cc88ffcb3fab03f3ffdd6170","created_at":"2026-08-28T11:42:22.542Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.559Z","job_size_bytes":19,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-cc88ffcb3fab03f3ffdd6170: done: 0.041509 sec","job_status":"done","queue_duration_s":3.939192,"scheduling_latency_s":3.939209,"redis_calls":11,"redis_duration_s":0.003192,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000357,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001611,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000271,"duration_s":0.041509,"completed_at":"2026-08-28T11:42:26.540Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236gitlab # [ 443.843006] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.542Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c2fa1286ac184e1bb52c35c2","created_at":"2026-08-28T11:42:22.566Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.579Z","job_size_bytes":66,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c2fa1286ac184e1bb52c35c2: start","job_status":"start","queue_duration_s":3.962673,"scheduling_latency_s":3.962696}2237gitlab # [ 443.861077] gitaly[907]: time="2026-08-28T11:42:26.520Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9MHSHX1XZTFT5GXFGTM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.510" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2238gitlab # [ 443.875886] gitlab-workhorse[820]: time="2026-08-28T11:42:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162239gitlab # [ 443.877943] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1012240gitlab # [ 443.879885] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.582Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c2fa1286ac184e1bb52c35c2","created_at":"2026-08-28T11:42:22.566Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.579Z","job_size_bytes":66,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c2fa1286ac184e1bb52c35c2: done: 0.039831 sec","job_status":"done","queue_duration_s":3.962673,"scheduling_latency_s":3.962696,"redis_calls":10,"redis_duration_s":0.002072,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000386,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029808,"duration_s":0.039831,"completed_at":"2026-08-28T11:42:26.582Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 443.926742] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.583Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4c9e24a92faeada496ca60a0","created_at":"2026-08-28T11:42:22.582Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.598Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4c9e24a92faeada496ca60a0: start","job_status":"start","queue_duration_s":3.985516,"scheduling_latency_s":3.985533}2242gitlab # [ 443.943305] nixbot[3630]: INFO: ::ffff:127.0.0.1:38754 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2243gitlab # [ 443.945312] gitaly[907]: time="2026-08-28T11:42:26.520Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9MHSHX1XZTFT5GXFGTM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.510" grpc.time_ms=9.947 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2244gitlab # [ 443.960155] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.636Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4c9e24a92faeada496ca60a0","created_at":"2026-08-28T11:42:22.582Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.598Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4c9e24a92faeada496ca60a0: done: 0.052914 sec","job_status":"done","queue_duration_s":3.985516,"scheduling_latency_s":3.985533,"redis_calls":10,"redis_duration_s":0.006206,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003747,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040562,"duration_s":0.052914,"completed_at":"2026-08-28T11:42:26.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 444.004788] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.638Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d0d6d66497d9ab8b54713206","created_at":"2026-08-28T11:42:22.614Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.624Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d0d6d66497d9ab8b54713206: start","job_status":"start","queue_duration_s":4.014333,"scheduling_latency_s":4.01436}2246gitlab # [ 444.021092] gitaly[907]: time="2026-08-28T11:42:26.648Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142V9R9HPMJ23J86K6DVDA2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.630" grpc.time_ms=17.21 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2247gitlab # [ 444.034240] gitaly[907]: time="2026-08-28T11:42:26.751Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M142V9TWXEKCAK3KRF5JJ4QT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.735" pid=907 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2248gitlab # [ 444.044459] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.693Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d0d6d66497d9ab8b54713206","created_at":"2026-08-28T11:42:22.614Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.624Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d0d6d66497d9ab8b54713206: done: 0.055186 sec","job_status":"done","queue_duration_s":4.014333,"scheduling_latency_s":4.01436,"redis_calls":10,"redis_duration_s":0.006737,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000946,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004873,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034044,"duration_s":0.055186,"completed_at":"2026-08-28T11:42:26.693Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2249gitlab # [ 444.089220] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.696Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3dd7b44e2c1138d0534dc373","created_at":"2026-08-28T11:42:22.625Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.635Z","job_size_bytes":6,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3dd7b44e2c1138d0534dc373: start","job_status":"start","queue_duration_s":4.06127,"scheduling_latency_s":4.061299}2250gitlab # [ 444.106475] gitaly[907]: time="2026-08-28T11:42:26.751Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9TWXEKCAK3KRF5JJ4QT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.735" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2251gitlab # [ 444.120362] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.728Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3dd7b44e2c1138d0534dc373","created_at":"2026-08-28T11:42:22.625Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.635Z","job_size_bytes":6,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3dd7b44e2c1138d0534dc373: done: 0.031711 sec","job_status":"done","queue_duration_s":4.06127,"scheduling_latency_s":4.061299,"redis_calls":10,"redis_duration_s":0.002542,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000599,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001387,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036878,"duration_s":0.031711,"completed_at":"2026-08-28T11:42:26.728Z","load_balancing_strategy":"primary","db_duration_s":0.003001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2252gitlab # [ 444.164870] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2cc7d6c8db4bc43112e74e62","created_at":"2026-08-28T11:42:22.636Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.645Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2cc7d6c8db4bc43112e74e62: start","job_status":"start","queue_duration_s":4.084878,"scheduling_latency_s":4.084908}2253gitlab # [ 444.180338] gitaly[907]: time="2026-08-28T11:42:26.753Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9TWXEKCAK3KRF5JJ4QT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:26.735" grpc.time_ms=17.154 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2254gitlab # [ 444.195541] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.793Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2cc7d6c8db4bc43112e74e62","created_at":"2026-08-28T11:42:22.636Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.645Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2cc7d6c8db4bc43112e74e62: done: 0.063245 sec","job_status":"done","queue_duration_s":4.084878,"scheduling_latency_s":4.084908,"redis_calls":10,"redis_duration_s":0.004513,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002424,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000703,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00105588,"duration_s":0.063245,"completed_at":"2026-08-28T11:42:26.793Z","load_balancing_strategy":"primary","db_duration_s":0.01173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255gitlab # [ 444.239994] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.795Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"58dac7d2526a53a362555946","created_at":"2026-08-28T11:42:22.657Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.667Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-58dac7d2526a53a362555946: start","job_status":"start","queue_duration_s":4.127426,"scheduling_latency_s":4.127447}2256gitlab # [ 444.256211] gitaly[907]: time="2026-08-28T11:42:26.871Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:56.069" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:26.869" grpc.time_ms=1.481 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2257gitlab # [ 444.269061] nixbot[3630]: {"ts": "2026-08-28T11:42:27.148Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 \"HTTP/1.1 201 Created\""}2258gitlab # [ 444.271801] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.901Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"58dac7d2526a53a362555946","created_at":"2026-08-28T11:42:22.657Z","trace_propagation_headers":{"sentry-trace":"8daaed4dc3064ef0a89b78826de3e1d3-546ddb534eff47f8","baggage":"sentry-trace_id=8daaed4dc3064ef0a89b78826de3e1d3,sentry-release=Unknown"},"correlation_id":"01M142RGF9D7SRF7HY0BSAQP1H","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.667Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-58dac7d2526a53a362555946: done: 0.106176 sec","job_status":"done","queue_duration_s":4.127426,"scheduling_latency_s":4.127447,"redis_calls":14,"redis_duration_s":0.009774,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000452,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005358,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003964,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046116,"duration_s":0.106176,"completed_at":"2026-08-28T11:42:26.901Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259gitlab # [ 444.316876] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.904Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"60efbb5f7b39b049d2eacbe9","created_at":"2026-08-28T11:42:22.953Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-bfc637f0c1fc4eb8","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.966Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-60efbb5f7b39b049d2eacbe9: start","job_status":"start","queue_duration_s":3.938435,"scheduling_latency_s":3.938457}2260gitlab # [ 444.332883] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3582261gitlab # [ 444.335395] gitaly[907]: time="2026-08-28T11:42:26.895Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-28T11:42:56.082" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:26.883" grpc.time_ms=11.38 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2262gitlab # [ 444.349601] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.969Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"60efbb5f7b39b049d2eacbe9","created_at":"2026-08-28T11:42:22.953Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-bfc637f0c1fc4eb8","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:22.966Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-60efbb5f7b39b049d2eacbe9: done: 0.065307 sec","job_status":"done","queue_duration_s":3.938435,"scheduling_latency_s":3.938457,"redis_calls":19,"redis_duration_s":0.00527,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00098,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000519,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00312,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01148666,"duration_s":0.065307,"completed_at":"2026-08-28T11:42:26.969Z","load_balancing_strategy":"primary","db_duration_s":0.01391,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263gitlab # [ 444.395165] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:26.971Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5a9c678759440f7d5a7d39cc"],"class":"WebHooks::LogExecutionWorker","jid":"e163bb76f64cac9ffe104b89","created_at":"2026-08-28T11:42:24.937Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-721782f897ce4574","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c57a840b5456acdf3dc14b7a5fa502ef8ad8c314206fbad34f78da5ea0f1faa8","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:24.952Z","job_size_bytes":2593,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e163bb76f64cac9ffe104b89: start","job_status":"start","queue_duration_s":2.01929,"scheduling_latency_s":2.019312}2264gitlab # [ 444.412228] nixbot[3630]: {"ts": "2026-08-28T11:42:27.167Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-f1bc3ee186b8-4"}2265gitlab # [ 444.414476] nixbot[3630]: {"ts": "2026-08-28T11:42:27.302Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-f1bc3ee186b8-4", "argv0": "bwrap"}2266gitlab # [ 444.416888] gitaly[907]: time="2026-08-28T11:42:26.923Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:36.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:26.904" grpc.time_ms=18.592 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2267gitlab # [ 444.430289] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.046Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5a9c678759440f7d5a7d39cc"],"class":"WebHooks::LogExecutionWorker","jid":"e163bb76f64cac9ffe104b89","created_at":"2026-08-28T11:42:24.937Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-721782f897ce4574","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c57a840b5456acdf3dc14b7a5fa502ef8ad8c314206fbad34f78da5ea0f1faa8","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:24.952Z","job_size_bytes":2593,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e163bb76f64cac9ffe104b89: done: 0.074535 sec","job_status":"done","queue_duration_s":2.01929,"scheduling_latency_s":2.019312,"redis_calls":10,"redis_duration_s":0.002627,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000294,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000366,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044604,"duration_s":0.074535,"completed_at":"2026-08-28T11:42:27.046Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2268gitlab # [ 444.474988] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.048Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4c5c837a7f458005e8e7abcb","created_at":"2026-08-28T11:42:25.887Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-ece81f98953447e1","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:25.898Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4c5c837a7f458005e8e7abcb: start","job_status":"start","queue_duration_s":1.14999,"scheduling_latency_s":1.150022}2269gitlab # [ 444.490774] gitaly[907]: time="2026-08-28T11:42:26.949Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-28T11:42:36.048" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:26.948" grpc.time_ms=0.458 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2270gitlab # [ 444.501916] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.354Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4c5c837a7f458005e8e7abcb","created_at":"2026-08-28T11:42:25.887Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-ece81f98953447e1","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:25.898Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3572,"job_status":"rescheduled","message":"PipelineProcessWorker JID-4c5c837a7f458005e8e7abcb: rescheduled"}2271gitlab # [ 444.517966] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.361Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4c5c837a7f458005e8e7abcb","created_at":"2026-08-28T11:42:25.887Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-ece81f98953447e1","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:25.898Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4c5c837a7f458005e8e7abcb: done: 0.313302 sec","job_status":"done","queue_duration_s":1.14999,"scheduling_latency_s":1.150022,"gitaly_calls":3,"gitaly_duration_s":0.127001,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.010841,"redis_read_bytes":151,"redis_write_bytes":18010,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000603,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002445,"redis_queues_read_bytes":29,"redis_queues_write_bytes":12309,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.00394,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00032,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000331,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001599,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.011,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.145305,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.2668774,"duration_s":0.313302,"completed_at":"2026-08-28T11:42:27.361Z","load_balancing_strategy":"primary","db_duration_s":0.039374,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2272gitlab # [ 444.568990] gitaly[907]: time="2026-08-28T11:42:26.971Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-28T11:42:36.055" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:26.956" grpc.time_ms=14.147 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2273gitlab # [ 444.582266] gitaly[907]: time="2026-08-28T11:42:27.012Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:27.011" grpc.time_ms=0.458 pid=907 span.kind=server system=grpc2274gitlab # [ 444.591959] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.363Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5354c510e30fd2259dbe44b8"],"class":"WebHooks::LogExecutionWorker","jid":"f36d520f5398baaf154d0834","created_at":"2026-08-28T11:42:26.081Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-6c9fcaae003c41c5","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:91397be373411b997a3aa03b896c34e1adb963da9554f9c863eaad0b1a9563ea","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:26.087Z","job_size_bytes":2592,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f36d520f5398baaf154d0834: start","job_status":"start","queue_duration_s":1.275552,"scheduling_latency_s":1.275573}2275gitlab # [ 444.609123] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.398Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5354c510e30fd2259dbe44b8"],"class":"WebHooks::LogExecutionWorker","jid":"f36d520f5398baaf154d0834","created_at":"2026-08-28T11:42:26.081Z","trace_propagation_headers":{"sentry-trace":"07c20917e20248a2849599db645e0001-6c9fcaae003c41c5","baggage":"sentry-trace_id=07c20917e20248a2849599db645e0001,sentry-release=Unknown"},"correlation_id":"01M142RF3WDRRZ0V4BT8HPXT7G","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:91397be373411b997a3aa03b896c34e1adb963da9554f9c863eaad0b1a9563ea","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:26.087Z","job_size_bytes":2592,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f36d520f5398baaf154d0834: done: 0.03515 sec","job_status":"done","queue_duration_s":1.275552,"scheduling_latency_s":1.275573,"redis_calls":7,"redis_duration_s":0.001706,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001327,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03515,"completed_at":"2026-08-28T11:42:27.398Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 444.652520] gitaly[907]: time="2026-08-28T11:42:27.036Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:27.035" grpc.time_ms=0.468 pid=907 span.kind=server system=grpc2277gitlab # [ 444.662019] gitaly[907]: time="2026-08-28T11:42:27.041Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K diskcache=1d616583-cfee-4c7a-89c7-e5ffbc08f726 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-28T11:42:36.074" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:26.974" pid=907 ref=refs/keep-around/f1bc3ee186b853356efea66d316b0d35f7e7dd26 remote_ip="2001:db8:1::1" revision=f1bc3ee186b853356efea66d316b0d35f7e7dd26 span.kind=server system=grpc username=root2278gitlab # [ 444.673451] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.400Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"193022da66c42a95d6f5c8aa","created_at":"2026-08-28T11:42:26.951Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-7400246170d544f2","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:26.959Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-193022da66c42a95d6f5c8aa: start","job_status":"start","queue_duration_s":0.440453,"scheduling_latency_s":0.440478}2279gitlab # [ 444.690332] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.538Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"193022da66c42a95d6f5c8aa","created_at":"2026-08-28T11:42:26.951Z","trace_propagation_headers":{"sentry-trace":"816268c5d35b4b24956ce53555c309c7-7400246170d544f2","baggage":"sentry-trace_id=816268c5d35b4b24956ce53555c309c7,sentry-release=Unknown"},"correlation_id":"01M142R9WA3FDZ5899KKJJ6494","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:26.959Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-193022da66c42a95d6f5c8aa: done: 0.138072 sec","job_status":"done","queue_duration_s":0.440453,"scheduling_latency_s":0.440478,"redis_calls":11,"redis_duration_s":0.00428,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00306,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00080724,"duration_s":0.138072,"completed_at":"2026-08-28T11:42:27.538Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2280gitlab # [ 444.736109] gitaly[907]: time="2026-08-28T11:42:27.042Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5018 command.oublock=8 command.real_time_ms=60 command.spawn_token_fork_ms=2 command.system_time_ms=35 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-28T11:42:36.074" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-28T11:42:26.974" grpc.time_ms=67.242 pid=907 ref=refs/keep-around/f1bc3ee186b853356efea66d316b0d35f7e7dd26 remote_ip="2001:db8:1::1" revision=f1bc3ee186b853356efea66d316b0d35f7e7dd26 span.kind=server system=grpc username=root2281gitlab # [ 444.751061] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.541Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"c4a6542e02a4ac102206ba04","created_at":"2026-08-28T11:42:27.047Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-489e904191e34663","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.051Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-c4a6542e02a4ac102206ba04: start","job_status":"start","queue_duration_s":0.490477,"scheduling_latency_s":0.490517}2282gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26'2283gitlab # [ 444.768372] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.610Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"c4a6542e02a4ac102206ba04","created_at":"2026-08-28T11:42:27.047Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-489e904191e34663","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.051Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-c4a6542e02a4ac102206ba04: done: 0.069197 sec","job_status":"done","queue_duration_s":0.490477,"scheduling_latency_s":0.490517,"redis_calls":7,"redis_duration_s":0.003154,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069197,"completed_at":"2026-08-28T11:42:27.610Z","load_balancing_strategy":"primary","db_duration_s":0.011562,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2284gitlab # [ 444.811937] gitaly[907]: time="2026-08-28T11:42:27.045Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:57.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:27.045" grpc.time_ms=0.314 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2285gitlab # [ 444.823754] gitaly[907]: time="2026-08-28T11:42:27.164Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:27.164" grpc.time_ms=0.43 pid=907 span.kind=server system=grpc2286gitlab # [ 444.833140] nix-daemon[3856]: accepted connection from pid 4508, user nixbot2287gitlab # [ 444.834038] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.613Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"923c5119d50f31d6a06bfb91","created_at":"2026-08-28T11:42:27.052Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-489e904191e34663","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.055Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-923c5119d50f31d6a06bfb91: start","job_status":"start","queue_duration_s":0.558294,"scheduling_latency_s":0.558328}2288gitlab # [ 444.850238] gitaly[907]: time="2026-08-28T11:42:27.210Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:27.210" grpc.time_ms=0.446 pid=907 span.kind=server system=grpc2289gitlab # [ 444.860331] gitaly[907]: time="2026-08-28T11:42:27.243Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:27.243" grpc.time_ms=0.48 pid=907 span.kind=server system=grpc2290gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26', in 0.08 seconds)2291(finished: subtest: push delivers the webhook, in 94.33 seconds)2292subtest: eval and build post commit statuses2293gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2294gitlab # [ 444.870250] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.730Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"923c5119d50f31d6a06bfb91","created_at":"2026-08-28T11:42:27.052Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-489e904191e34663","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.055Z","job_size_bytes":65,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-923c5119d50f31d6a06bfb91: done: 0.117034 sec","job_status":"done","queue_duration_s":0.558294,"scheduling_latency_s":0.558328,"redis_calls":10,"redis_duration_s":0.0048850000000000005,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.002733,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.117034,"completed_at":"2026-08-28T11:42:27.730Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2295gitlab # [ 444.914236] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.733Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a70488314d143feb89412551","created_at":"2026-08-28T11:42:27.120Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-489e904191e34663","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.125Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a70488314d143feb89412551: start","job_status":"start","queue_duration_s":0.608038,"scheduling_latency_s":0.608074}2296gitlab # [ 444.931767] gitaly[907]: time="2026-08-28T11:42:27.252Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 diskcache=9a9025f7-ceb5-4a1d-8824-da481be29684 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:42:57.036" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:27.137" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2297gitlab # [ 444.941944] gitaly[907]: time="2026-08-28T11:42:27.253Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=99 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7153 command.oublock=0 command.real_time_ms=108 command.spawn_token_fork_ms=0 command.system_time_ms=49 command.user_time_ms=50 component=gitaly.UnaryServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:42:57.036" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:27.137" grpc.time_ms=115.492 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2298gitlab # [ 444.955501] nixbot[3630]: INFO: ::ffff:127.0.0.1:38756 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 200 OK2299gitlab # [ 444.957966] gitaly[907]: time="2026-08-28T11:42:27.312Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-28T11:42:57.110" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:27.310" grpc.time_ms=1.064 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2300gitlab # [ 444.971149] gitaly[907]: time="2026-08-28T11:42:27.345Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V8RRSB78PEGBXQAR2H61 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:57.143" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:27.344" grpc.time_ms=1.124 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2301gitlab # [ 445.009576] gitaly[907]: time="2026-08-28T11:42:27.903Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VAZ2KQTEEHYE9GEDZFZM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:57.900" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:27.901" grpc.time_ms=1.367 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2302gitlab # [ 445.022658] gitaly[907]: time="2026-08-28T11:42:27.908Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-28T11:42:57.706" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:27.906" grpc.time_ms=1.299 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2303gitlab # [ 445.067999] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1492304gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.19 seconds)2305gitlab # [ 445.082846] gitaly[907]: time="2026-08-28T11:42:27.976Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142V9Z78YV1XNVNTWTC795K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:57.774" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:27.975" grpc.time_ms=0.992 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2306gitlab # [ 445.102073] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.995Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a70488314d143feb89412551","created_at":"2026-08-28T11:42:27.120Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-489e904191e34663","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.125Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a70488314d143feb89412551: done: 0.261352 sec","job_status":"done","queue_duration_s":0.608038,"scheduling_latency_s":0.608074,"gitaly_calls":2,"gitaly_duration_s":0.011256,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.017163,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001659,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001726,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000643,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007592,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000514,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000735,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.194448,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1804182,"duration_s":0.261352,"completed_at":"2026-08-28T11:42:27.994Z","load_balancing_strategy":"primary","db_duration_s":0.049926,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2307gitlab # [ 445.156037] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:27.997Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"44702f2557704f7274077be3","created_at":"2026-08-28T11:42:27.127Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.130Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-44702f2557704f7274077be3: start","job_status":"start","queue_duration_s":0.867158,"scheduling_latency_s":0.867189}2308gitlab # [ 445.173002] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.058Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"44702f2557704f7274077be3","created_at":"2026-08-28T11:42:27.127Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.130Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-44702f2557704f7274077be3: done: 0.061211 sec","job_status":"done","queue_duration_s":0.867158,"scheduling_latency_s":0.867189,"redis_calls":7,"redis_duration_s":0.0024000000000000002,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061211,"completed_at":"2026-08-28T11:42:28.058Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014192,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2309gitlab # [ 445.216092] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.061Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d051ac43aa9b07c86f2a880f","created_at":"2026-08-28T11:42:27.260Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.264Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d051ac43aa9b07c86f2a880f: start","job_status":"start","queue_duration_s":0.796924,"scheduling_latency_s":0.796956}2310gitlab # [ 445.232742] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.102Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d051ac43aa9b07c86f2a880f","created_at":"2026-08-28T11:42:27.260Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.264Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d051ac43aa9b07c86f2a880f: done: 0.041078 sec","job_status":"done","queue_duration_s":0.796924,"scheduling_latency_s":0.796956,"redis_calls":7,"redis_duration_s":0.002344,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000671,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041078,"completed_at":"2026-08-28T11:42:28.102Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2311gitlab # [ 445.275963] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.104Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"1cf93c937aeb341d7c686a86","created_at":"2026-08-28T11:42:27.272Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.274Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-1cf93c937aeb341d7c686a86: start","job_status":"start","queue_duration_s":0.829587,"scheduling_latency_s":0.829616}2312gitlab # [ 445.291868] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.143Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"1cf93c937aeb341d7c686a86","created_at":"2026-08-28T11:42:27.272Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.274Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-1cf93c937aeb341d7c686a86: done: 0.038884 sec","job_status":"done","queue_duration_s":0.829587,"scheduling_latency_s":0.829616,"redis_calls":7,"redis_duration_s":0.002042,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001479,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038884,"completed_at":"2026-08-28T11:42:28.143Z","load_balancing_strategy":"primary","db_duration_s":0.008631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2313gitlab # [ 445.335455] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.145Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fe811afc34d523415ac7e70c","created_at":"2026-08-28T11:42:27.282Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.284Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fe811afc34d523415ac7e70c: start","job_status":"start","queue_duration_s":0.860313,"scheduling_latency_s":0.860342}2314gitlab # [ 445.351189] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.197Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fe811afc34d523415ac7e70c","created_at":"2026-08-28T11:42:27.282Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.284Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fe811afc34d523415ac7e70c: done: 0.05263 sec","job_status":"done","queue_duration_s":0.860313,"scheduling_latency_s":0.860342,"redis_calls":10,"redis_duration_s":0.012348,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.003636,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002891,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05263,"completed_at":"2026-08-28T11:42:28.197Z","load_balancing_strategy":"primary","db_duration_s":0.012289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2315gitlab # [ 445.394829] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.199Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7f68620cb6f5afa20d50c027","created_at":"2026-08-28T11:42:27.285Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.287Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7f68620cb6f5afa20d50c027: start","job_status":"start","queue_duration_s":0.912514,"scheduling_latency_s":0.912545}2316gitlab # [ 445.411689] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.233Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"7f68620cb6f5afa20d50c027","created_at":"2026-08-28T11:42:27.285Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.287Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-7f68620cb6f5afa20d50c027: done: 0.033764 sec","job_status":"done","queue_duration_s":0.912514,"scheduling_latency_s":0.912545,"redis_calls":7,"redis_duration_s":0.002062,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033764,"completed_at":"2026-08-28T11:42:28.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2317gitlab # [ 445.454865] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.235Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"07ec0358058f12d66d5df743","created_at":"2026-08-28T11:42:27.287Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.290Z","job_size_bytes":19,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-07ec0358058f12d66d5df743: start","job_status":"start","queue_duration_s":0.94513,"scheduling_latency_s":0.945157}2318gitlab # [ 445.471758] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"07ec0358058f12d66d5df743","created_at":"2026-08-28T11:42:27.287Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.290Z","job_size_bytes":19,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-07ec0358058f12d66d5df743: done: 0.031742 sec","job_status":"done","queue_duration_s":0.94513,"scheduling_latency_s":0.945157,"redis_calls":8,"redis_duration_s":0.0024809999999999997,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001927,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000554,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031742,"completed_at":"2026-08-28T11:42:28.267Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2319gitlab # [ 445.516168] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.269Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ebcd3563bb6bd93f28926d20","created_at":"2026-08-28T11:42:27.291Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.293Z","job_size_bytes":66,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ebcd3563bb6bd93f28926d20: start","job_status":"start","queue_duration_s":0.975806,"scheduling_latency_s":0.975835}2320gitlab # [ 445.533304] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.296Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ebcd3563bb6bd93f28926d20","created_at":"2026-08-28T11:42:27.291Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.293Z","job_size_bytes":66,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ebcd3563bb6bd93f28926d20: done: 0.027297 sec","job_status":"done","queue_duration_s":0.975806,"scheduling_latency_s":0.975835,"redis_calls":7,"redis_duration_s":0.0019130000000000002,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027297,"completed_at":"2026-08-28T11:42:28.296Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321gitlab # [ 445.578594] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.298Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4b96cfc66c154170c711f44b","created_at":"2026-08-28T11:42:27.293Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.295Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4b96cfc66c154170c711f44b: start","job_status":"start","queue_duration_s":1.002289,"scheduling_latency_s":1.002315}2322gitlab # [ 445.595210] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.352Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4b96cfc66c154170c711f44b","created_at":"2026-08-28T11:42:27.293Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.295Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4b96cfc66c154170c711f44b: done: 0.054771 sec","job_status":"done","queue_duration_s":1.002289,"scheduling_latency_s":1.002315,"redis_calls":7,"redis_duration_s":0.003432,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002671,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054771,"completed_at":"2026-08-28T11:42:28.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2323gitlab # [ 445.638358] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.356Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7f650edecb519d3c06be7090","created_at":"2026-08-28T11:42:27.296Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.298Z","job_size_bytes":6,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7f650edecb519d3c06be7090: start","job_status":"start","queue_duration_s":1.058734,"scheduling_latency_s":1.058781}2324gitlab # [ 445.654599] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.390Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7f650edecb519d3c06be7090","created_at":"2026-08-28T11:42:27.296Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.298Z","job_size_bytes":6,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7f650edecb519d3c06be7090: done: 0.033481 sec","job_status":"done","queue_duration_s":1.058734,"scheduling_latency_s":1.058781,"redis_calls":7,"redis_duration_s":0.002803,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033481,"completed_at":"2026-08-28T11:42:28.390Z","load_balancing_strategy":"primary","db_duration_s":0.00083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2325gitlab # [ 445.696937] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.393Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bb9e16b04f39ffb1bcf2d2dc","created_at":"2026-08-28T11:42:27.298Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.300Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bb9e16b04f39ffb1bcf2d2dc: start","job_status":"start","queue_duration_s":1.092788,"scheduling_latency_s":1.092826}2326gitlab # [ 445.712694] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.435Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bb9e16b04f39ffb1bcf2d2dc","created_at":"2026-08-28T11:42:27.298Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.300Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bb9e16b04f39ffb1bcf2d2dc: done: 0.042944 sec","job_status":"done","queue_duration_s":1.092788,"scheduling_latency_s":1.092826,"redis_calls":7,"redis_duration_s":0.0024619999999999998,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042944,"completed_at":"2026-08-28T11:42:28.435Z","load_balancing_strategy":"primary","db_duration_s":0.006242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2327gitlab # [ 445.756372] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.439Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a49113244eb443ec7cc262a3","created_at":"2026-08-28T11:42:27.355Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.356Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a49113244eb443ec7cc262a3: start","job_status":"start","queue_duration_s":1.082849,"scheduling_latency_s":1.082901}2328gitlab # [ 445.771909] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.505Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a49113244eb443ec7cc262a3","created_at":"2026-08-28T11:42:27.355Z","trace_propagation_headers":{"sentry-trace":"6185064287e24d4ea878127527a3bdd5-4222445bdcf14719","baggage":"sentry-trace_id=6185064287e24d4ea878127527a3bdd5,sentry-release=Unknown"},"correlation_id":"01M142V8RRSB78PEGBXQAR2H61","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.356Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a49113244eb443ec7cc262a3: done: 0.065497 sec","job_status":"done","queue_duration_s":1.082849,"scheduling_latency_s":1.082901,"redis_calls":8,"redis_duration_s":0.003469,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002709,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065497,"completed_at":"2026-08-28T11:42:28.504Z","load_balancing_strategy":"primary","db_duration_s":0.008471,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2329gitlab # [ 445.815039] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.508Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3fd3bc4f4d4fe2577f34775b","created_at":"2026-08-28T11:42:27.875Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-817f1dffb4c04b0d","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.879Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3fd3bc4f4d4fe2577f34775b: start","job_status":"start","queue_duration_s":0.629009,"scheduling_latency_s":0.629048}2330gitlab # [ 445.831641] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:28.621Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3fd3bc4f4d4fe2577f34775b","created_at":"2026-08-28T11:42:27.875Z","trace_propagation_headers":{"sentry-trace":"5fcf276e1c65415f8c38e814e6dd26ee-817f1dffb4c04b0d","baggage":"sentry-trace_id=5fcf276e1c65415f8c38e814e6dd26ee,sentry-release=Unknown"},"correlation_id":"01M142V9Z78YV1XNVNTWTC795K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:27.879Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3fd3bc4f4d4fe2577f34775b: done: 0.113754 sec","job_status":"done","queue_duration_s":0.629009,"scheduling_latency_s":0.629048,"redis_calls":7,"redis_duration_s":0.002467,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00068,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.113754,"completed_at":"2026-08-28T11:42:28.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2331gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2332gitlab # [ 446.259316] gitaly[907]: time="2026-08-28T11:42:29.151Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VC5T64CX386WVBWH636S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:42:59.149" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:29.150" grpc.time_ms=1.241 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2333gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.23 seconds)2334gitlab # [ 446.307016] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1482335gitlab # [ 446.821950] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:29.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"01d9cb7de03e116f879de357","created_at":"2026-08-28T11:42:29.702Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"099bc0781445992b414a1d49d9707c64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-28T11:42:29.710Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-01d9cb7de03e116f879de357: start","job_status":"start","queue_duration_s":0.003725,"scheduling_latency_s":0.003785}2336gitlab # [ 446.990001] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:29.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"01d9cb7de03e116f879de357","created_at":"2026-08-28T11:42:29.702Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"099bc0781445992b414a1d49d9707c64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-28T11:42:29.710Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-01d9cb7de03e116f879de357: done: 0.168478 sec","job_status":"done","queue_duration_s":0.003725,"scheduling_latency_s":0.003785,"redis_calls":9,"redis_duration_s":0.017639000000000002,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002679,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002844,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.168478,"completed_at":"2026-08-28T11:42:29.882Z","load_balancing_strategy":"primary","db_duration_s":0.026472,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2337gitlab # [ 447.074546] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:29.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d65b12ff5e5f44ccc72a4c97","created_at":"2026-08-28T11:42:29.724Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.751Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d65b12ff5e5f44ccc72a4c97: start","job_status":"start","queue_duration_s":0.136584,"scheduling_latency_s":0.136633}2338gitlab # [ 447.170956] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:30.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d65b12ff5e5f44ccc72a4c97","created_at":"2026-08-28T11:42:29.724Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.751Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d65b12ff5e5f44ccc72a4c97: done: 0.174838 sec","job_status":"done","queue_duration_s":0.136584,"scheduling_latency_s":0.136633,"redis_calls":18,"redis_duration_s":0.033727,"redis_read_bytes":19,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006435,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.002326,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.018903,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006063,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.174838,"completed_at":"2026-08-28T11:42:30.062Z","load_balancing_strategy":"primary","db_duration_s":0.012218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2339gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2340gitlab # [ 447.272590] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:30.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e15b2a8ca60e2b9e3b2a6783","created_at":"2026-08-28T11:42:29.764Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d3994a0de19c70989bea1d7b2846eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.793Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e15b2a8ca60e2b9e3b2a6783: start","job_status":"start","queue_duration_s":0.27994,"scheduling_latency_s":0.279989}2341gitlab # [ 447.316087] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:30.165Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2342gitlab # [ 447.326543] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:30.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e15b2a8ca60e2b9e3b2a6783","created_at":"2026-08-28T11:42:29.764Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d3994a0de19c70989bea1d7b2846eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.793Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e15b2a8ca60e2b9e3b2a6783: done: 0.132138 sec","job_status":"done","queue_duration_s":0.27994,"scheduling_latency_s":0.279989,"redis_calls":9,"redis_duration_s":0.022064,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004802,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012036,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005226,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.132138,"completed_at":"2026-08-28T11:42:30.205Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2343gitlab # [ 447.428810] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:30.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"27d951168746fee8f5c97e02","created_at":"2026-08-28T11:42:29.838Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.912Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-27d951168746fee8f5c97e02: start","job_status":"start","queue_duration_s":0.35031,"scheduling_latency_s":0.350356}2344gitlab # [ 447.454502] gitaly[907]: time="2026-08-28T11:42:30.335Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VDBSD28QFNDYR2D07E9J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:00.333" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:30.334" grpc.time_ms=0.854 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2345gitlab # [ 447.476864] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1032346gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.17 seconds)2347gitlab # [ 448.146054] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"27d951168746fee8f5c97e02","created_at":"2026-08-28T11:42:29.838Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.912Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-27d951168746fee8f5c97e02: done: 0.774366 sec","job_status":"done","queue_duration_s":0.35031,"scheduling_latency_s":0.350356,"redis_calls":70,"redis_duration_s":0.257706,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.015793,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001785,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.238086,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.302292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.774366,"completed_at":"2026-08-28T11:42:31.037Z","load_balancing_strategy":"primary","db_duration_s":0.052036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2348gitlab # [ 448.153620] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.046Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2ebd752d2b6567212ab95e41","created_at":"2026-08-28T11:42:29.936Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35edfbc593e16783b9d8e10d8085c857","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.963Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2ebd752d2b6567212ab95e41: start","job_status":"start","queue_duration_s":1.083583,"scheduling_latency_s":1.083632}2349gitlab # [ 448.267263] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.159Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2ebd752d2b6567212ab95e41","created_at":"2026-08-28T11:42:29.936Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35edfbc593e16783b9d8e10d8085c857","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:29.963Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2ebd752d2b6567212ab95e41: done: 0.113319 sec","job_status":"done","queue_duration_s":1.083583,"scheduling_latency_s":1.083632,"redis_calls":11,"redis_duration_s":0.023584,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.019453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00309,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00152754,"exclusive_lock_hold_duration_s":0.00770712,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.113319,"completed_at":"2026-08-28T11:42:31.159Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2350gitlab # [ 448.372860] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.163Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dc1adec09affadbe8f3aeaa9","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dc1adec09affadbe8f3aeaa9: start","job_status":"start","queue_duration_s":1.15308,"scheduling_latency_s":1.153118}2351gitlab # [ 448.385950] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.194Z","message":"No existing merge request to be cleaned up."}2352gitlab # [ 448.387622] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.228Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dc1adec09affadbe8f3aeaa9","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dc1adec09affadbe8f3aeaa9: done: 0.06563 sec","job_status":"done","queue_duration_s":1.15308,"scheduling_latency_s":1.153118,"redis_calls":12,"redis_duration_s":0.004599,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00069,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002277,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06563,"completed_at":"2026-08-28T11:42:31.228Z","load_balancing_strategy":"primary","db_duration_s":0.013435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2353gitlab # [ 448.428146] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.231Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9cd5f257044837eea2cf05c9","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9cd5f257044837eea2cf05c9: start","job_status":"start","queue_duration_s":1.221291,"scheduling_latency_s":1.221328}2354gitlab # [ 448.440114] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.251Z","message":"No existing merge request to be cleaned up."}2355gitlab # [ 448.441790] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.265Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9cd5f257044837eea2cf05c9","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9cd5f257044837eea2cf05c9: done: 0.034672 sec","job_status":"done","queue_duration_s":1.221291,"scheduling_latency_s":1.221328,"redis_calls":10,"redis_duration_s":0.003465,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002081,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034672,"completed_at":"2026-08-28T11:42:31.265Z","load_balancing_strategy":"primary","db_duration_s":0.003467,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2356gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2357gitlab # [ 448.481215] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.268Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d3b0a9e60b374dadd96a5570","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d3b0a9e60b374dadd96a5570: start","job_status":"start","queue_duration_s":1.258459,"scheduling_latency_s":1.258488}2358gitlab # [ 448.492849] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.285Z","message":"No existing merge request to be cleaned up."}2359gitlab # [ 448.494481] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.299Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d3b0a9e60b374dadd96a5570","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d3b0a9e60b374dadd96a5570: done: 0.030877 sec","job_status":"done","queue_duration_s":1.258459,"scheduling_latency_s":1.258488,"redis_calls":10,"redis_duration_s":0.003663,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002324,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030877,"completed_at":"2026-08-28T11:42:31.299Z","load_balancing_strategy":"primary","db_duration_s":0.003121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2360gitlab # [ 448.532807] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.301Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f508eecb6f40efd9c141948","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f508eecb6f40efd9c141948: start","job_status":"start","queue_duration_s":1.29153,"scheduling_latency_s":1.291558}2361gitlab # [ 448.545044] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.317Z","message":"No existing merge request to be cleaned up."}2362gitlab # [ 448.546664] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.329Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f508eecb6f40efd9c141948","created_at":"2026-08-28T11:42:29.950Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-b0e75997df39410f","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d44c80c4362f68893d6fc42e5fcee0c3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.009Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f508eecb6f40efd9c141948: done: 0.027728 sec","job_status":"done","queue_duration_s":1.29153,"scheduling_latency_s":1.291558,"redis_calls":10,"redis_duration_s":0.0030080000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027728,"completed_at":"2026-08-28T11:42:31.329Z","load_balancing_strategy":"primary","db_duration_s":0.00299,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2363gitlab # [ 448.585371] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.331Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"b632f85a9d77e760e624351d","created_at":"2026-08-28T11:42:29.999Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4011d1d36b9bda834877b2d887b8447","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.018Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-b632f85a9d77e760e624351d: start","job_status":"start","queue_duration_s":1.312715,"scheduling_latency_s":1.31274}2364gitlab # [ 448.598378] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.355Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"b632f85a9d77e760e624351d","created_at":"2026-08-28T11:42:29.999Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4011d1d36b9bda834877b2d887b8447","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.018Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-b632f85a9d77e760e624351d: done: 0.024452 sec","job_status":"done","queue_duration_s":1.312715,"scheduling_latency_s":1.31274,"redis_calls":11,"redis_duration_s":0.003112,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000541,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000651,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024452,"completed_at":"2026-08-28T11:42:31.355Z","load_balancing_strategy":"primary","db_duration_s":0.000835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2365gitlab # [ 448.641193] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.357Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9ca9c68607ab6472242d1f90","created_at":"2026-08-28T11:42:30.049Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca528649c5a0c15ed1dfe8338dafdacf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.070Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9ca9c68607ab6472242d1f90: start","job_status":"start","queue_duration_s":1.286418,"scheduling_latency_s":1.286444}2366gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.20 seconds)2367gitlab # [ 448.655421] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.416Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"9ca9c68607ab6472242d1f90","created_at":"2026-08-28T11:42:30.049Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca528649c5a0c15ed1dfe8338dafdacf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.070Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-9ca9c68607ab6472242d1f90: done: 0.059024 sec","job_status":"done","queue_duration_s":1.286418,"scheduling_latency_s":1.286444,"redis_calls":9,"redis_duration_s":0.002977,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000581,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001768,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059024,"completed_at":"2026-08-28T11:42:31.416Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2368gitlab # [ 448.698352] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a8aa9de97b9bd05234c80615","created_at":"2026-08-28T11:42:30.081Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b3a9b1c03dcd5de39745fdd73f2f260","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.096Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a8aa9de97b9bd05234c80615: start","job_status":"start","queue_duration_s":1.323285,"scheduling_latency_s":1.323333}2369gitlab # [ 448.712698] gitaly[907]: time="2026-08-28T11:42:31.520Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VEG0NAE1AANTTG9YCA54 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:01.518" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:31.518" grpc.time_ms=1.062 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2370gitlab # [ 448.725802] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1392371gitlab # [ 448.968416] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.861Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"a8aa9de97b9bd05234c80615","created_at":"2026-08-28T11:42:30.081Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b3a9b1c03dcd5de39745fdd73f2f260","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.096Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-a8aa9de97b9bd05234c80615: done: 0.441665 sec","job_status":"done","queue_duration_s":1.323285,"scheduling_latency_s":1.323333,"redis_calls":13,"redis_duration_s":0.0066879999999999995,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001782,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001845,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003061,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.067,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.067,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.347338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00152148,"exclusive_lock_hold_duration_s":0.390654,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.441665,"completed_at":"2026-08-28T11:42:31.861Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.105192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2372gitlab # [ 449.070411] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c6e1ebccef2f1b3e910ae9fb","created_at":"2026-08-28T11:42:30.118Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad16b755cd0b9feef1d11ade0ee7e872","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.146Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c6e1ebccef2f1b3e910ae9fb: start","job_status":"start","queue_duration_s":1.717512,"scheduling_latency_s":1.717542}2373gitlab # [ 449.096487] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c6e1ebccef2f1b3e910ae9fb","created_at":"2026-08-28T11:42:30.118Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad16b755cd0b9feef1d11ade0ee7e872","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.146Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c6e1ebccef2f1b3e910ae9fb: done: 0.068862 sec","job_status":"done","queue_duration_s":1.717512,"scheduling_latency_s":1.717542,"redis_calls":11,"redis_duration_s":0.004395,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00146332,"exclusive_lock_hold_duration_s":0.02573422,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.068862,"completed_at":"2026-08-28T11:42:31.932Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2374gitlab # [ 449.151934] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2ad7be55256cbaeb7d794809","created_at":"2026-08-28T11:42:30.162Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4662d9fd7613c6afb4f9f79989c7c451","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.198Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2ad7be55256cbaeb7d794809: start","job_status":"start","queue_duration_s":1.737304,"scheduling_latency_s":1.737332}2375gitlab # [ 449.166823] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.955Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2376gitlab # [ 449.170270] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2ad7be55256cbaeb7d794809","created_at":"2026-08-28T11:42:30.162Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4662d9fd7613c6afb4f9f79989c7c451","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.198Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2ad7be55256cbaeb7d794809: done: 0.030613 sec","job_status":"done","queue_duration_s":1.737304,"scheduling_latency_s":1.737332,"redis_calls":9,"redis_duration_s":0.0028710000000000003,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000633,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000648,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030613,"completed_at":"2026-08-28T11:42:31.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2377gitlab # [ 449.212515] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:31.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9e2eaa35eeed57abe66fcc4a","created_at":"2026-08-28T11:42:30.255Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94a1c1da898e00243bb4e2c28597cc8c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.269Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9e2eaa35eeed57abe66fcc4a: start","job_status":"start","queue_duration_s":1.698763,"scheduling_latency_s":1.698787}2378gitlab # [ 449.226402] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.022Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9e2eaa35eeed57abe66fcc4a","created_at":"2026-08-28T11:42:30.255Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94a1c1da898e00243bb4e2c28597cc8c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.269Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9e2eaa35eeed57abe66fcc4a: done: 0.054472 sec","job_status":"done","queue_duration_s":1.698763,"scheduling_latency_s":1.698787,"redis_calls":9,"redis_duration_s":0.0030930000000000003,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.054472,"completed_at":"2026-08-28T11:42:32.022Z","load_balancing_strategy":"primary","db_duration_s":0.020803,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2379gitlab # [ 449.268897] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.025Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a13417331fcfb62cdbb89fba","created_at":"2026-08-28T11:42:30.293Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d051fb622c6edc6c4767091ddc557ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.305Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a13417331fcfb62cdbb89fba: start","job_status":"start","queue_duration_s":1.720398,"scheduling_latency_s":1.720427}2380gitlab # [ 449.284143] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.043Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2381gitlab # [ 449.287217] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.054Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a13417331fcfb62cdbb89fba","created_at":"2026-08-28T11:42:30.293Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d051fb622c6edc6c4767091ddc557ab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.305Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a13417331fcfb62cdbb89fba: done: 0.028547 sec","job_status":"done","queue_duration_s":1.720398,"scheduling_latency_s":1.720427,"redis_calls":9,"redis_duration_s":0.0028669999999999998,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000615,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028547,"completed_at":"2026-08-28T11:42:32.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000733,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2382gitlab # [ 449.329926] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4931be72907ed6cca90a252a","created_at":"2026-08-28T11:42:30.325Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9036fc50d4639ba7bf2035e2c832949","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.375Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4931be72907ed6cca90a252a: start","job_status":"start","queue_duration_s":1.680869,"scheduling_latency_s":1.680891}2383gitlab # [ 449.344483] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.124Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4931be72907ed6cca90a252a","created_at":"2026-08-28T11:42:30.325Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9036fc50d4639ba7bf2035e2c832949","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.375Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4931be72907ed6cca90a252a: done: 0.068505 sec","job_status":"done","queue_duration_s":1.680869,"scheduling_latency_s":1.680891,"redis_calls":13,"redis_duration_s":0.004227,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001176,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001602,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00069252,"exclusive_lock_hold_duration_s":0.0374667,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.068505,"completed_at":"2026-08-28T11:42:32.124Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2384gitlab # [ 449.389198] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.126Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3c574a45f6d3c570f6839231","created_at":"2026-08-28T11:42:30.402Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"faad551916add977dbf7552630c52bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.417Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3c574a45f6d3c570f6839231: start","job_status":"start","queue_duration_s":1.708783,"scheduling_latency_s":1.708807}2385gitlab # [ 449.403947] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"3c574a45f6d3c570f6839231","created_at":"2026-08-28T11:42:30.402Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"faad551916add977dbf7552630c52bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.417Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-3c574a45f6d3c570f6839231: done: 0.029503 sec","job_status":"done","queue_duration_s":1.708783,"scheduling_latency_s":1.708807,"redis_calls":11,"redis_duration_s":0.003423,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001268,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029503,"completed_at":"2026-08-28T11:42:32.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2386gitlab # [ 449.446545] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0ab7fc5cabf7c28b758bea00","created_at":"2026-08-28T11:42:30.442Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3934d3fb132065dc53c52cab7b257ae1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.475Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0ab7fc5cabf7c28b758bea00: start","job_status":"start","queue_duration_s":1.682817,"scheduling_latency_s":1.68284}2387gitlab # [ 449.460141] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.189Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0ab7fc5cabf7c28b758bea00","created_at":"2026-08-28T11:42:30.442Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3934d3fb132065dc53c52cab7b257ae1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.475Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0ab7fc5cabf7c28b758bea00: done: 0.031219 sec","job_status":"done","queue_duration_s":1.682817,"scheduling_latency_s":1.68284,"redis_calls":11,"redis_duration_s":0.003006,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00061114,"exclusive_lock_hold_duration_s":0.0090238,"duration_s":0.031219,"completed_at":"2026-08-28T11:42:32.189Z","load_balancing_strategy":"primary","db_duration_s":0.005036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2388gitlab # [ 449.502471] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"36dad42cb4f5736e47c492c8","created_at":"2026-08-28T11:42:30.534Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d3d047a8d4b451eef8ec95c6c794bfc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.560Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-36dad42cb4f5736e47c492c8: start","job_status":"start","queue_duration_s":1.630808,"scheduling_latency_s":1.63083}2389gitlab # [ 449.516499] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"36dad42cb4f5736e47c492c8","created_at":"2026-08-28T11:42:30.534Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d3d047a8d4b451eef8ec95c6c794bfc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.560Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-36dad42cb4f5736e47c492c8: done: 0.028142 sec","job_status":"done","queue_duration_s":1.630808,"scheduling_latency_s":1.63083,"redis_calls":9,"redis_duration_s":0.0024990000000000004,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000497,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001422,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028142,"completed_at":"2026-08-28T11:42:32.219Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2390gitlab # [ 449.558299] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"01724c073445e6809fb7ed56","created_at":"2026-08-28T11:42:30.590Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2eaf4f4fec6e94f97f6b74219f5eae0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.621Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-01724c073445e6809fb7ed56: start","job_status":"start","queue_duration_s":1.599235,"scheduling_latency_s":1.599254}2391gitlab # [ 449.573783] nix-daemon[3856]: accepted connection from pid 4552, user nixbot2392gitlab # [ 449.574714] nix-daemon[3856]: accepted connection from pid 4559, user nixbot2393gitlab # [ 449.575651] nix-daemon[3856]: accepted connection from pid 4508, user nixbot2394gitlab # [ 449.576664] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"01724c073445e6809fb7ed56","created_at":"2026-08-28T11:42:30.590Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2eaf4f4fec6e94f97f6b74219f5eae0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.621Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-01724c073445e6809fb7ed56: done: 0.033366 sec","job_status":"done","queue_duration_s":1.599235,"scheduling_latency_s":1.599254,"redis_calls":12,"redis_duration_s":0.003326,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001546,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01004884,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.033366,"completed_at":"2026-08-28T11:42:32.254Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004544,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2395gitlab # [ 449.620542] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.255Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"08de54a1b9f1432f11a866df","created_at":"2026-08-28T11:42:30.677Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fb3979a23711d81266a4500a2a77d7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-28T11:42:30.702Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-08de54a1b9f1432f11a866df: start","job_status":"start","queue_duration_s":1.553258,"scheduling_latency_s":1.553274}2396gitlab # [ 449.633803] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.267Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2397gitlab # [ 449.637078] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"08de54a1b9f1432f11a866df","created_at":"2026-08-28T11:42:30.677Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fb3979a23711d81266a4500a2a77d7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-28T11:42:30.702Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-08de54a1b9f1432f11a866df: done: 0.018219 sec","job_status":"done","queue_duration_s":1.553258,"scheduling_latency_s":1.553274,"redis_calls":9,"redis_duration_s":0.002202,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000499,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018219,"completed_at":"2026-08-28T11:42:32.273Z","load_balancing_strategy":"primary","db_duration_s":0.000592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2398gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2399gitlab # [ 449.677946] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fcfae83a9a316d57b4f8f155","created_at":"2026-08-28T11:42:30.725Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0f030101a008e381a41a7863c3e887e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.749Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fcfae83a9a316d57b4f8f155: start","job_status":"start","queue_duration_s":1.525511,"scheduling_latency_s":1.525528}2400gitlab # [ 449.691719] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.286Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2401gitlab # [ 449.696154] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fcfae83a9a316d57b4f8f155","created_at":"2026-08-28T11:42:30.725Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0f030101a008e381a41a7863c3e887e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.749Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fcfae83a9a316d57b4f8f155: done: 0.017886 sec","job_status":"done","queue_duration_s":1.525511,"scheduling_latency_s":1.525528,"redis_calls":9,"redis_duration_s":0.002219,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017886,"completed_at":"2026-08-28T11:42:32.293Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2402gitlab # [ 449.736737] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1ba39e2be048907e53bb0e68","created_at":"2026-08-28T11:42:30.774Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7bc702c5296388415aca0ef7f5293ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-28T11:42:30.795Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1ba39e2be048907e53bb0e68: start","job_status":"start","queue_duration_s":1.498814,"scheduling_latency_s":1.49883}2403gitlab # [ 449.749866] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.315Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2404gitlab # [ 449.753529] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.463Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1ba39e2be048907e53bb0e68","created_at":"2026-08-28T11:42:30.774Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7bc702c5296388415aca0ef7f5293ca4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-28T11:42:30.795Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1ba39e2be048907e53bb0e68: done: 0.168287 sec","job_status":"done","queue_duration_s":1.498814,"scheduling_latency_s":1.49883,"redis_calls":9,"redis_duration_s":0.0064069999999999995,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.151178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.168287,"completed_at":"2026-08-28T11:42:32.462Z","load_balancing_strategy":"primary","db_duration_s":0.000693,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2405gitlab # [ 449.794455] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6788f68cd5e4680534408b84","created_at":"2026-08-28T11:42:30.824Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4152bc926e9292167f6cfb2546bff1de","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.842Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6788f68cd5e4680534408b84: start","job_status":"start","queue_duration_s":1.622033,"scheduling_latency_s":1.622058}2406gitlab # [ 449.809086] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.497Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6788f68cd5e4680534408b84","created_at":"2026-08-28T11:42:30.824Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4152bc926e9292167f6cfb2546bff1de","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.842Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6788f68cd5e4680534408b84: done: 0.032505 sec","job_status":"done","queue_duration_s":1.622033,"scheduling_latency_s":1.622058,"redis_calls":9,"redis_duration_s":0.002052,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032505,"completed_at":"2026-08-28T11:42:32.497Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2407gitlab # [ 449.851532] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cc7dc2e7019c711f10ded74e","created_at":"2026-08-28T11:42:30.896Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a16622fbbb447bd0c8771d74295c14e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.932Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cc7dc2e7019c711f10ded74e: start","job_status":"start","queue_duration_s":1.566258,"scheduling_latency_s":1.566274}2408gitlab # [ 449.866205] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.520Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cc7dc2e7019c711f10ded74e","created_at":"2026-08-28T11:42:30.896Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a16622fbbb447bd0c8771d74295c14e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.932Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cc7dc2e7019c711f10ded74e: done: 0.021556 sec","job_status":"done","queue_duration_s":1.566258,"scheduling_latency_s":1.566274,"redis_calls":9,"redis_duration_s":0.00244,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000595,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001308,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021556,"completed_at":"2026-08-28T11:42:32.520Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2409gitlab # [ 449.908103] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e5f6535143e538d9a9ab0db3","created_at":"2026-08-28T11:42:30.957Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0ac739597a6c1da192f7edd380db5b6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.982Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e5f6535143e538d9a9ab0db3: start","job_status":"start","queue_duration_s":1.539244,"scheduling_latency_s":1.539261}2410gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.27 seconds)2411gitlab # [ 449.923551] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e5f6535143e538d9a9ab0db3","created_at":"2026-08-28T11:42:30.957Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0ac739597a6c1da192f7edd380db5b6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.982Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e5f6535143e538d9a9ab0db3: done: 0.027347 sec","job_status":"done","queue_duration_s":1.539244,"scheduling_latency_s":1.539261,"redis_calls":11,"redis_duration_s":0.003043,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000525,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00061634,"exclusive_lock_hold_duration_s":0.00586556,"duration_s":0.027347,"completed_at":"2026-08-28T11:42:32.549Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2412gitlab # [ 449.966002] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ebbcf9c9f3f960c5b8416d6","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ebbcf9c9f3f960c5b8416d6: start","job_status":"start","queue_duration_s":1.562776,"scheduling_latency_s":1.562792}2413gitlab # [ 449.978680] gitaly[907]: time="2026-08-28T11:42:32.769Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VFPKBSC7DQJBSR2R03TJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:02.067" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:32.768" grpc.time_ms=1.359 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2414gitlab # [ 449.991871] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1732415gitlab # [ 449.994681] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ebbcf9c9f3f960c5b8416d6","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ebbcf9c9f3f960c5b8416d6: done: 0.022894 sec","job_status":"done","queue_duration_s":1.562776,"scheduling_latency_s":1.562792,"redis_calls":15,"redis_duration_s":0.003488,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022894,"completed_at":"2026-08-28T11:42:32.573Z","load_balancing_strategy":"primary","db_duration_s":0.000616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2416gitlab # [ 450.035896] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dc7fafb001efc2d0a79be50","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dc7fafb001efc2d0a79be50: start","job_status":"start","queue_duration_s":1.587096,"scheduling_latency_s":1.587111}2417gitlab # [ 450.048732] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dc7fafb001efc2d0a79be50","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dc7fafb001efc2d0a79be50: done: 0.016168 sec","job_status":"done","queue_duration_s":1.587096,"scheduling_latency_s":1.587111,"redis_calls":13,"redis_duration_s":0.003006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016168,"completed_at":"2026-08-28T11:42:32.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2418gitlab # [ 450.088083] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e7222ac3a4040f2f05fd959","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e7222ac3a4040f2f05fd959: start","job_status":"start","queue_duration_s":1.604686,"scheduling_latency_s":1.604701}2419gitlab # [ 450.100428] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e7222ac3a4040f2f05fd959","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e7222ac3a4040f2f05fd959: done: 0.016507 sec","job_status":"done","queue_duration_s":1.604686,"scheduling_latency_s":1.604701,"redis_calls":13,"redis_duration_s":0.0032730000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002305,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016507,"completed_at":"2026-08-28T11:42:32.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2420gitlab # [ 450.140569] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2211c4819b2d2ba689683cb","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2211c4819b2d2ba689683cb: start","job_status":"start","queue_duration_s":1.623537,"scheduling_latency_s":1.623553}2421gitlab # [ 450.153154] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2211c4819b2d2ba689683cb","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2211c4819b2d2ba689683cb: done: 0.018826 sec","job_status":"done","queue_duration_s":1.623537,"scheduling_latency_s":1.623553,"redis_calls":13,"redis_duration_s":0.003416,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002354,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018826,"completed_at":"2026-08-28T11:42:32.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2422gitlab # [ 450.192723] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af65628b07635ef2d403f904","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af65628b07635ef2d403f904: start","job_status":"start","queue_duration_s":1.646701,"scheduling_latency_s":1.646717}2423gitlab # [ 450.205116] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af65628b07635ef2d403f904","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af65628b07635ef2d403f904: done: 0.018777 sec","job_status":"done","queue_duration_s":1.646701,"scheduling_latency_s":1.646717,"redis_calls":13,"redis_duration_s":0.004623,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018777,"completed_at":"2026-08-28T11:42:32.653Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2424gitlab # [ 450.244370] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b2761c5b94ef76145ce9271","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b2761c5b94ef76145ce9271: start","job_status":"start","queue_duration_s":1.666728,"scheduling_latency_s":1.666743}2425gitlab # [ 450.256693] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b2761c5b94ef76145ce9271","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b2761c5b94ef76145ce9271: done: 0.015316 sec","job_status":"done","queue_duration_s":1.666728,"scheduling_latency_s":1.666743,"redis_calls":13,"redis_duration_s":0.0018449999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015316,"completed_at":"2026-08-28T11:42:32.670Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2426gitlab # [ 450.295833] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1802771e0257ee84c3890c59","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1802771e0257ee84c3890c59: start","job_status":"start","queue_duration_s":1.683871,"scheduling_latency_s":1.683887}2427gitlab # [ 450.308508] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.687Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1802771e0257ee84c3890c59","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1802771e0257ee84c3890c59: done: 0.015093 sec","job_status":"done","queue_duration_s":1.683871,"scheduling_latency_s":1.683887,"redis_calls":13,"redis_duration_s":0.001662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015093,"completed_at":"2026-08-28T11:42:32.687Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2428gitlab # [ 450.348410] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2728b090575d4e8f17e521f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2728b090575d4e8f17e521f: start","job_status":"start","queue_duration_s":1.700076,"scheduling_latency_s":1.700092}2429gitlab # [ 450.360580] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2728b090575d4e8f17e521f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2728b090575d4e8f17e521f: done: 0.01541 sec","job_status":"done","queue_duration_s":1.700076,"scheduling_latency_s":1.700092,"redis_calls":13,"redis_duration_s":0.001927,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01541,"completed_at":"2026-08-28T11:42:32.703Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2430gitlab # [ 450.399956] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6d65ab45376dfb09c413c46","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6d65ab45376dfb09c413c46: start","job_status":"start","queue_duration_s":1.717435,"scheduling_latency_s":1.717452}2431gitlab # [ 450.412463] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6d65ab45376dfb09c413c46","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6d65ab45376dfb09c413c46: done: 0.015689 sec","job_status":"done","queue_duration_s":1.717435,"scheduling_latency_s":1.717452,"redis_calls":13,"redis_duration_s":0.001784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015689,"completed_at":"2026-08-28T11:42:32.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2432gitlab # [ 450.451691] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d3109ee826ba427ee912032","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d3109ee826ba427ee912032: start","job_status":"start","queue_duration_s":1.73431,"scheduling_latency_s":1.734326}2433gitlab # [ 450.464389] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d3109ee826ba427ee912032","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d3109ee826ba427ee912032: done: 0.016075 sec","job_status":"done","queue_duration_s":1.73431,"scheduling_latency_s":1.734326,"redis_calls":13,"redis_duration_s":0.001882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016075,"completed_at":"2026-08-28T11:42:32.738Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2434gitlab # [ 450.503656] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"049b68bdf1c2259b20f7fc43","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-049b68bdf1c2259b20f7fc43: start","job_status":"start","queue_duration_s":1.751552,"scheduling_latency_s":1.751568}2435gitlab # [ 450.516252] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"049b68bdf1c2259b20f7fc43","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-049b68bdf1c2259b20f7fc43: done: 0.016302 sec","job_status":"done","queue_duration_s":1.751552,"scheduling_latency_s":1.751568,"redis_calls":13,"redis_duration_s":0.0018210000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016302,"completed_at":"2026-08-28T11:42:32.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2436gitlab # [ 450.556649] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3b36a05d7c8b95efa35bfb4","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3b36a05d7c8b95efa35bfb4: start","job_status":"start","queue_duration_s":1.769157,"scheduling_latency_s":1.769172}2437gitlab # [ 450.569002] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3b36a05d7c8b95efa35bfb4","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3b36a05d7c8b95efa35bfb4: done: 0.015606 sec","job_status":"done","queue_duration_s":1.769157,"scheduling_latency_s":1.769172,"redis_calls":13,"redis_duration_s":0.001737,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015606,"completed_at":"2026-08-28T11:42:32.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2438gitlab # [ 450.608177] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98de44bba7254fc0dc9c2089","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98de44bba7254fc0dc9c2089: start","job_status":"start","queue_duration_s":1.785903,"scheduling_latency_s":1.785919}2439gitlab # [ 450.620436] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98de44bba7254fc0dc9c2089","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98de44bba7254fc0dc9c2089: done: 0.014965 sec","job_status":"done","queue_duration_s":1.785903,"scheduling_latency_s":1.785919,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014965,"completed_at":"2026-08-28T11:42:32.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2440gitlab # [ 450.659221] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"186547b11625242812833a5e","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-186547b11625242812833a5e: start","job_status":"start","queue_duration_s":1.802804,"scheduling_latency_s":1.802819}2441gitlab # [ 450.671775] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"186547b11625242812833a5e","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-186547b11625242812833a5e: done: 0.015389 sec","job_status":"done","queue_duration_s":1.802804,"scheduling_latency_s":1.802819,"redis_calls":13,"redis_duration_s":0.0017670000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015389,"completed_at":"2026-08-28T11:42:32.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2442gitlab # [ 450.711308] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1c6fec6db297878aca455ec","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1c6fec6db297878aca455ec: start","job_status":"start","queue_duration_s":1.819268,"scheduling_latency_s":1.819283}2443gitlab # [ 450.723645] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1c6fec6db297878aca455ec","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1c6fec6db297878aca455ec: done: 0.015948 sec","job_status":"done","queue_duration_s":1.819268,"scheduling_latency_s":1.819283,"redis_calls":13,"redis_duration_s":0.002913,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001074,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001839,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015948,"completed_at":"2026-08-28T11:42:32.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2444gitlab # [ 450.764231] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"512cad516827f83bd27539a5","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-512cad516827f83bd27539a5: start","job_status":"start","queue_duration_s":1.837344,"scheduling_latency_s":1.837359}2445gitlab # [ 450.776902] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"512cad516827f83bd27539a5","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-512cad516827f83bd27539a5: done: 0.015291 sec","job_status":"done","queue_duration_s":1.837344,"scheduling_latency_s":1.837359,"redis_calls":13,"redis_duration_s":0.00255,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015291,"completed_at":"2026-08-28T11:42:32.840Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2446gitlab # [ 450.815947] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78639de4cc8915495819d73c","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78639de4cc8915495819d73c: start","job_status":"start","queue_duration_s":1.853799,"scheduling_latency_s":1.853815}2447gitlab # [ 450.828590] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78639de4cc8915495819d73c","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78639de4cc8915495819d73c: done: 0.014949 sec","job_status":"done","queue_duration_s":1.853799,"scheduling_latency_s":1.853815,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014949,"completed_at":"2026-08-28T11:42:32.856Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2448gitlab # [ 450.868108] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"707a63941463b1dea929bac4","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-707a63941463b1dea929bac4: start","job_status":"start","queue_duration_s":1.871074,"scheduling_latency_s":1.87109}2449gitlab # [ 450.880488] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"707a63941463b1dea929bac4","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-707a63941463b1dea929bac4: done: 0.015828 sec","job_status":"done","queue_duration_s":1.871074,"scheduling_latency_s":1.87109,"redis_calls":13,"redis_duration_s":0.0029300000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015828,"completed_at":"2026-08-28T11:42:32.875Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2450gitlab # [ 450.919904] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b273133261371cfce7bc39ec","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b273133261371cfce7bc39ec: start","job_status":"start","queue_duration_s":1.888271,"scheduling_latency_s":1.888287}2451gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2452gitlab # [ 450.933020] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b273133261371cfce7bc39ec","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b273133261371cfce7bc39ec: done: 0.016771 sec","job_status":"done","queue_duration_s":1.888271,"scheduling_latency_s":1.888287,"redis_calls":13,"redis_duration_s":0.002836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016771,"completed_at":"2026-08-28T11:42:32.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2453gitlab # [ 450.972091] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf6b1e2b815a69dd0106d9b","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf6b1e2b815a69dd0106d9b: start","job_status":"start","queue_duration_s":1.906502,"scheduling_latency_s":1.906518}2454gitlab # [ 450.983406] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf6b1e2b815a69dd0106d9b","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf6b1e2b815a69dd0106d9b: done: 0.016167 sec","job_status":"done","queue_duration_s":1.906502,"scheduling_latency_s":1.906518,"redis_calls":13,"redis_duration_s":0.003476,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016167,"completed_at":"2026-08-28T11:42:32.910Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2455gitlab # [ 451.022254] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f76ca1485d875749496d12da","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f76ca1485d875749496d12da: start","job_status":"start","queue_duration_s":1.924018,"scheduling_latency_s":1.924034}2456gitlab # [ 451.034890] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f76ca1485d875749496d12da","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f76ca1485d875749496d12da: done: 0.015936 sec","job_status":"done","queue_duration_s":1.924018,"scheduling_latency_s":1.924034,"redis_calls":13,"redis_duration_s":0.002856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015936,"completed_at":"2026-08-28T11:42:32.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2457gitlab # [ 451.074392] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffb48d335c60c5936304a8e9","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffb48d335c60c5936304a8e9: start","job_status":"start","queue_duration_s":1.941318,"scheduling_latency_s":1.941333}2458gitlab # [ 451.087027] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffb48d335c60c5936304a8e9","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffb48d335c60c5936304a8e9: done: 0.015591 sec","job_status":"done","queue_duration_s":1.941318,"scheduling_latency_s":1.941333,"redis_calls":13,"redis_duration_s":0.002615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001643,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015591,"completed_at":"2026-08-28T11:42:32.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2459gitlab # [ 451.126818] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daff471bae8baa41893303d2","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daff471bae8baa41893303d2: start","job_status":"start","queue_duration_s":1.958305,"scheduling_latency_s":1.95832}2460gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.21 seconds)2461gitlab # [ 451.139493] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daff471bae8baa41893303d2","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daff471bae8baa41893303d2: done: 0.015947 sec","job_status":"done","queue_duration_s":1.958305,"scheduling_latency_s":1.95832,"redis_calls":13,"redis_duration_s":0.002801,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015947,"completed_at":"2026-08-28T11:42:32.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2462gitlab # [ 451.180461] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b969e1a3bc50e9dc9a77e489","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b969e1a3bc50e9dc9a77e489: start","job_status":"start","queue_duration_s":1.975649,"scheduling_latency_s":1.975664}2463gitlab # [ 451.193348] gitaly[907]: time="2026-08-28T11:42:34.011Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VGYJPATNTGPAWZCPMJKP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:04.008" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:34.009" grpc.time_ms=1.359 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2464gitlab # [ 451.206680] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1082465gitlab # [ 451.209488] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b969e1a3bc50e9dc9a77e489","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b969e1a3bc50e9dc9a77e489: done: 0.015531 sec","job_status":"done","queue_duration_s":1.975649,"scheduling_latency_s":1.975664,"redis_calls":13,"redis_duration_s":0.002834,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015531,"completed_at":"2026-08-28T11:42:32.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2466gitlab # [ 451.249039] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99dc37f73d1fd1d029084c7b","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99dc37f73d1fd1d029084c7b: start","job_status":"start","queue_duration_s":1.992548,"scheduling_latency_s":1.992564}2467gitlab # [ 451.261587] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99dc37f73d1fd1d029084c7b","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99dc37f73d1fd1d029084c7b: done: 0.015408 sec","job_status":"done","queue_duration_s":1.992548,"scheduling_latency_s":1.992564,"redis_calls":13,"redis_duration_s":0.003046,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015408,"completed_at":"2026-08-28T11:42:32.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2468gitlab # [ 451.301470] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:32.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0af0cbcf82398daa5aef6461","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0af0cbcf82398daa5aef6461: start","job_status":"start","queue_duration_s":2.009404,"scheduling_latency_s":2.009418}2469gitlab # [ 451.313957] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0af0cbcf82398daa5aef6461","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0af0cbcf82398daa5aef6461: done: 0.015454 sec","job_status":"done","queue_duration_s":2.009404,"scheduling_latency_s":2.009418,"redis_calls":13,"redis_duration_s":0.002895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015454,"completed_at":"2026-08-28T11:42:33.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2470gitlab # [ 451.353237] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"992a1f5e6722aa49559f5e32","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-992a1f5e6722aa49559f5e32: start","job_status":"start","queue_duration_s":2.026254,"scheduling_latency_s":2.026269}2471gitlab # [ 451.365421] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"992a1f5e6722aa49559f5e32","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-992a1f5e6722aa49559f5e32: done: 0.015678 sec","job_status":"done","queue_duration_s":2.026254,"scheduling_latency_s":2.026269,"redis_calls":13,"redis_duration_s":0.0027530000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015678,"completed_at":"2026-08-28T11:42:33.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2472gitlab # [ 451.405645] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a15d89a38bfbb3d77506873","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a15d89a38bfbb3d77506873: start","job_status":"start","queue_duration_s":2.043294,"scheduling_latency_s":2.043309}2473gitlab # [ 451.418026] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a15d89a38bfbb3d77506873","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a15d89a38bfbb3d77506873: done: 0.015189 sec","job_status":"done","queue_duration_s":2.043294,"scheduling_latency_s":2.043309,"redis_calls":13,"redis_duration_s":0.002867,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015189,"completed_at":"2026-08-28T11:42:33.046Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2474gitlab # [ 451.456996] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ba92a1f26ce6e94f5269efd","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ba92a1f26ce6e94f5269efd: start","job_status":"start","queue_duration_s":2.059905,"scheduling_latency_s":2.059919}2475gitlab # [ 451.469359] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ba92a1f26ce6e94f5269efd","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ba92a1f26ce6e94f5269efd: done: 0.015215 sec","job_status":"done","queue_duration_s":2.059905,"scheduling_latency_s":2.059919,"redis_calls":13,"redis_duration_s":0.002665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001653,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015215,"completed_at":"2026-08-28T11:42:33.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2476gitlab # [ 451.508365] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"948158ff96d990cfa7aaa2e1","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-948158ff96d990cfa7aaa2e1: start","job_status":"start","queue_duration_s":2.076552,"scheduling_latency_s":2.076566}2477gitlab # [ 451.520680] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"948158ff96d990cfa7aaa2e1","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-948158ff96d990cfa7aaa2e1: done: 0.015638 sec","job_status":"done","queue_duration_s":2.076552,"scheduling_latency_s":2.076566,"redis_calls":13,"redis_duration_s":0.0028380000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015638,"completed_at":"2026-08-28T11:42:33.080Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2478gitlab # [ 451.559936] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c3cc80ebd82d9b789c84874","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c3cc80ebd82d9b789c84874: start","job_status":"start","queue_duration_s":2.093471,"scheduling_latency_s":2.093486}2479gitlab # [ 451.573533] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c3cc80ebd82d9b789c84874","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c3cc80ebd82d9b789c84874: done: 0.01497 sec","job_status":"done","queue_duration_s":2.093471,"scheduling_latency_s":2.093486,"redis_calls":13,"redis_duration_s":0.002627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001667,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01497,"completed_at":"2026-08-28T11:42:33.096Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2480gitlab # [ 451.612425] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c11b6790034584705408bb41","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c11b6790034584705408bb41: start","job_status":"start","queue_duration_s":2.109747,"scheduling_latency_s":2.109762}2481gitlab # [ 451.624761] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c11b6790034584705408bb41","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c11b6790034584705408bb41: done: 0.014981 sec","job_status":"done","queue_duration_s":2.109747,"scheduling_latency_s":2.109762,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001795,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014981,"completed_at":"2026-08-28T11:42:33.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2482gitlab # [ 451.664254] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b0b73b5f544d4f189111aca","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b0b73b5f544d4f189111aca: start","job_status":"start","queue_duration_s":2.126173,"scheduling_latency_s":2.126188}2483gitlab # [ 451.676502] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b0b73b5f544d4f189111aca","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b0b73b5f544d4f189111aca: done: 0.015106 sec","job_status":"done","queue_duration_s":2.126173,"scheduling_latency_s":2.126188,"redis_calls":13,"redis_duration_s":0.002725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015106,"completed_at":"2026-08-28T11:42:33.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2484gitlab # [ 451.715600] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb22feaf60bb1ec7af9768e2","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb22feaf60bb1ec7af9768e2: start","job_status":"start","queue_duration_s":2.143438,"scheduling_latency_s":2.143454}2485gitlab # [ 451.728226] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb22feaf60bb1ec7af9768e2","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb22feaf60bb1ec7af9768e2: done: 0.015614 sec","job_status":"done","queue_duration_s":2.143438,"scheduling_latency_s":2.143454,"redis_calls":13,"redis_duration_s":0.00282,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015614,"completed_at":"2026-08-28T11:42:33.147Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2486gitlab # [ 451.767297] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a03b13d37868faadd9648c8","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a03b13d37868faadd9648c8: start","job_status":"start","queue_duration_s":2.160409,"scheduling_latency_s":2.160423}2487gitlab # [ 451.779782] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a03b13d37868faadd9648c8","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a03b13d37868faadd9648c8: done: 0.01535 sec","job_status":"done","queue_duration_s":2.160409,"scheduling_latency_s":2.160423,"redis_calls":13,"redis_duration_s":0.002733,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01535,"completed_at":"2026-08-28T11:42:33.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2488gitlab # [ 451.820428] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8feeaed60e26c971e931d2ac","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8feeaed60e26c971e931d2ac: start","job_status":"start","queue_duration_s":2.177674,"scheduling_latency_s":2.177689}2489gitlab # [ 451.832727] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8feeaed60e26c971e931d2ac","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8feeaed60e26c971e931d2ac: done: 0.015213 sec","job_status":"done","queue_duration_s":2.177674,"scheduling_latency_s":2.177689,"redis_calls":13,"redis_duration_s":0.0025510000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015213,"completed_at":"2026-08-28T11:42:33.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2490gitlab # [ 451.871780] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b79320f410fd0ab7734934a8","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b79320f410fd0ab7734934a8: start","job_status":"start","queue_duration_s":2.194267,"scheduling_latency_s":2.194281}2491gitlab # [ 451.884346] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b79320f410fd0ab7734934a8","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b79320f410fd0ab7734934a8: done: 0.015419 sec","job_status":"done","queue_duration_s":2.194267,"scheduling_latency_s":2.194281,"redis_calls":13,"redis_duration_s":0.0027730000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015419,"completed_at":"2026-08-28T11:42:33.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2492gitlab # [ 451.923539] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fdf8c1d095e49c2dda68478","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fdf8c1d095e49c2dda68478: start","job_status":"start","queue_duration_s":2.211081,"scheduling_latency_s":2.211095}2493gitlab # [ 451.936134] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fdf8c1d095e49c2dda68478","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fdf8c1d095e49c2dda68478: done: 0.015659 sec","job_status":"done","queue_duration_s":2.211081,"scheduling_latency_s":2.211095,"redis_calls":13,"redis_duration_s":0.0029500000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015659,"completed_at":"2026-08-28T11:42:33.214Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2494gitlab # [ 451.975416] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5943efea86e5e19af29a99f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5943efea86e5e19af29a99f: start","job_status":"start","queue_duration_s":2.228108,"scheduling_latency_s":2.228123}2495gitlab # [ 451.988023] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5943efea86e5e19af29a99f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5943efea86e5e19af29a99f: done: 0.015566 sec","job_status":"done","queue_duration_s":2.228108,"scheduling_latency_s":2.228123,"redis_calls":13,"redis_duration_s":0.002739,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001803,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015566,"completed_at":"2026-08-28T11:42:33.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2496gitlab # [ 452.028344] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"144f0e5ed9268eb837ae3f0f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-144f0e5ed9268eb837ae3f0f: start","job_status":"start","queue_duration_s":2.245037,"scheduling_latency_s":2.245052}2497gitlab # [ 452.040647] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"144f0e5ed9268eb837ae3f0f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-144f0e5ed9268eb837ae3f0f: done: 0.015091 sec","job_status":"done","queue_duration_s":2.245037,"scheduling_latency_s":2.245052,"redis_calls":13,"redis_duration_s":0.002834,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015091,"completed_at":"2026-08-28T11:42:33.248Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2498gitlab # [ 452.079679] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f7ef34fa4414dbab3199a4","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f7ef34fa4414dbab3199a4: start","job_status":"start","queue_duration_s":2.261872,"scheduling_latency_s":2.261887}2499gitlab # [ 452.092226] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9f7ef34fa4414dbab3199a4","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9f7ef34fa4414dbab3199a4: done: 0.014957 sec","job_status":"done","queue_duration_s":2.261872,"scheduling_latency_s":2.261887,"redis_calls":13,"redis_duration_s":0.002626,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001696,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014957,"completed_at":"2026-08-28T11:42:33.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2500gitlab # [ 452.131059] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"367087d528334628dbe1e71f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-367087d528334628dbe1e71f: start","job_status":"start","queue_duration_s":2.278181,"scheduling_latency_s":2.278196}2501gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2502gitlab # [ 452.143632] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"367087d528334628dbe1e71f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-367087d528334628dbe1e71f: done: 0.015291 sec","job_status":"done","queue_duration_s":2.278181,"scheduling_latency_s":2.278196,"redis_calls":13,"redis_duration_s":0.00272,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001791,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015291,"completed_at":"2026-08-28T11:42:33.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2503gitlab # [ 452.182229] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4e1cd8c9f519718fda94582","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4e1cd8c9f519718fda94582: start","job_status":"start","queue_duration_s":2.29508,"scheduling_latency_s":2.295094}2504gitlab # [ 452.193704] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4e1cd8c9f519718fda94582","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4e1cd8c9f519718fda94582: done: 0.015265 sec","job_status":"done","queue_duration_s":2.29508,"scheduling_latency_s":2.295094,"redis_calls":13,"redis_duration_s":0.002833,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001859,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015265,"completed_at":"2026-08-28T11:42:33.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2505gitlab # [ 452.232932] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7fcabed86c4ab1ba5ad1f3c","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7fcabed86c4ab1ba5ad1f3c: start","job_status":"start","queue_duration_s":2.311865,"scheduling_latency_s":2.311879}2506gitlab # [ 452.245278] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7fcabed86c4ab1ba5ad1f3c","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7fcabed86c4ab1ba5ad1f3c: done: 0.015267 sec","job_status":"done","queue_duration_s":2.311865,"scheduling_latency_s":2.311879,"redis_calls":13,"redis_duration_s":0.002963,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015267,"completed_at":"2026-08-28T11:42:33.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2507gitlab # [ 452.284304] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc9e53bf9e53ba92c031eb5","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc9e53bf9e53ba92c031eb5: start","job_status":"start","queue_duration_s":2.328509,"scheduling_latency_s":2.328524}2508gitlab # [ 452.296608] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc9e53bf9e53ba92c031eb5","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc9e53bf9e53ba92c031eb5: done: 0.015162 sec","job_status":"done","queue_duration_s":2.328509,"scheduling_latency_s":2.328524,"redis_calls":13,"redis_duration_s":0.00284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015162,"completed_at":"2026-08-28T11:42:33.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2509gitlab # [ 452.335752] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09bd5596410c596573375ea3","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09bd5596410c596573375ea3: start","job_status":"start","queue_duration_s":2.345047,"scheduling_latency_s":2.345061}2510gitlab # [ 452.348337] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09bd5596410c596573375ea3","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09bd5596410c596573375ea3: done: 0.015178 sec","job_status":"done","queue_duration_s":2.345047,"scheduling_latency_s":2.345061,"redis_calls":13,"redis_duration_s":0.002767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015178,"completed_at":"2026-08-28T11:42:33.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2511gitlab # [ 452.387743] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3addae418d43ce619440843","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3addae418d43ce619440843: start","job_status":"start","queue_duration_s":2.361501,"scheduling_latency_s":2.361516}2512gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.26 seconds)2513gitlab # [ 452.400391] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3addae418d43ce619440843","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3addae418d43ce619440843: done: 0.015039 sec","job_status":"done","queue_duration_s":2.361501,"scheduling_latency_s":2.361516,"redis_calls":13,"redis_duration_s":0.002695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001727,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015039,"completed_at":"2026-08-28T11:42:33.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2514gitlab # [ 452.440825] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ab794d68604599a1b6a5972","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ab794d68604599a1b6a5972: start","job_status":"start","queue_duration_s":2.377834,"scheduling_latency_s":2.377848}2515gitlab # [ 452.454540] gitaly[907]: time="2026-08-28T11:42:35.258Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VJ4DNG859AY5J52ZV6H8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:05.256" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:35.257" grpc.time_ms=1.276 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2516gitlab # [ 452.467930] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1582517gitlab # [ 452.470816] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ab794d68604599a1b6a5972","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ab794d68604599a1b6a5972: done: 0.02515 sec","job_status":"done","queue_duration_s":2.377834,"scheduling_latency_s":2.377848,"redis_calls":13,"redis_duration_s":0.0038799999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02515,"completed_at":"2026-08-28T11:42:33.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2518gitlab # [ 452.510685] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb53313dddf4a4709b3229fb","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb53313dddf4a4709b3229fb: start","job_status":"start","queue_duration_s":2.406682,"scheduling_latency_s":2.406721}2519gitlab # [ 452.523965] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb53313dddf4a4709b3229fb","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb53313dddf4a4709b3229fb: done: 0.036963 sec","job_status":"done","queue_duration_s":2.406682,"scheduling_latency_s":2.406721,"redis_calls":13,"redis_duration_s":0.0035989999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002317,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036963,"completed_at":"2026-08-28T11:42:33.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2520gitlab # [ 452.565004] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7578b17dc3eb5dd505a798f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7578b17dc3eb5dd505a798f: start","job_status":"start","queue_duration_s":2.445931,"scheduling_latency_s":2.44596}2521gitlab # [ 452.577738] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7578b17dc3eb5dd505a798f","created_at":"2026-08-28T11:42:30.429Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-50254edd294b4c30","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"501a30f0d76e008ccad033bcb9119a7f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-28T11:42:30.988Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7578b17dc3eb5dd505a798f: done: 0.026275 sec","job_status":"done","queue_duration_s":2.445931,"scheduling_latency_s":2.44596,"redis_calls":13,"redis_duration_s":0.0031869999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026275,"completed_at":"2026-08-28T11:42:33.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2522gitlab # [ 452.620710] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:33.462Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ffd1ca54e46876cf8f2af9fd","created_at":"2026-08-28T11:42:31.014Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ffccb13528b2d82e3d8fd60f994a809","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:31.041Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ffd1ca54e46876cf8f2af9fd: start","job_status":"start","queue_duration_s":2.421021,"scheduling_latency_s":2.421043}2523gitlab # [ 452.636655] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:34.399Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ffd1ca54e46876cf8f2af9fd","created_at":"2026-08-28T11:42:31.014Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ffccb13528b2d82e3d8fd60f994a809","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:31.041Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ffd1ca54e46876cf8f2af9fd: done: 0.937434 sec","job_status":"done","queue_duration_s":2.421021,"scheduling_latency_s":2.421043,"redis_calls":564,"redis_duration_s":0.164131,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000665,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004546,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.15892,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.561118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.937434,"completed_at":"2026-08-28T11:42:34.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2524gitlab # [ 452.681322] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:34.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"5901484d32dc07dca5fdcc3a","created_at":"2026-08-28T11:42:31.058Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e100ad9ca9cfa62e6b4a2b1a23f627ee","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:31.075Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-5901484d32dc07dca5fdcc3a: start","job_status":"start","queue_duration_s":3.327993,"scheduling_latency_s":3.328045}2525gitlab # [ 452.696334] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:34.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"5901484d32dc07dca5fdcc3a","created_at":"2026-08-28T11:42:31.058Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e100ad9ca9cfa62e6b4a2b1a23f627ee","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:31.075Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-5901484d32dc07dca5fdcc3a: done: 0.246019 sec","job_status":"done","queue_duration_s":3.327993,"scheduling_latency_s":3.328045,"redis_calls":12,"redis_duration_s":0.004938,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000902,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001996,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.222586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0014374,"exclusive_lock_hold_duration_s":0.19753268000000002,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.246019,"completed_at":"2026-08-28T11:42:34.649Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011087,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}2526gitlab # [ 452.742123] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:34.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"d648df487f7cf98ea0cc64fd","created_at":"2026-08-28T11:42:31.097Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f55e5bb804c909ad50862deaed38946","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:31.115Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-d648df487f7cf98ea0cc64fd: start","job_status":"start","queue_duration_s":3.539793,"scheduling_latency_s":3.539819}2527gitlab # [ 452.757177] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:34.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"d648df487f7cf98ea0cc64fd","created_at":"2026-08-28T11:42:31.097Z","trace_propagation_headers":{"sentry-trace":"28fad9c1626a4bac803b47d39120092a-2a7f8d50ed73450c","baggage":"sentry-trace_id=28fad9c1626a4bac803b47d39120092a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f55e5bb804c909ad50862deaed38946","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:31.115Z","job_size_bytes":2,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-d648df487f7cf98ea0cc64fd: done: 0.038682 sec","job_status":"done","queue_duration_s":3.539793,"scheduling_latency_s":3.539819,"redis_calls":12,"redis_duration_s":0.003662,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000743,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035694,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.038682,"completed_at":"2026-08-28T11:42:34.693Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2528gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2529gitlab # [ 453.617769] gitaly[907]: time="2026-08-28T11:42:36.510Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VKB59SNT3ZRCTJ8AMNF2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:06.407" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:36.508" grpc.time_ms=1.38 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2530gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.26 seconds)2531gitlab # [ 453.683922] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1862532gitlab # [ 453.729324] gitaly[907]: time="2026-08-28T11:42:36.622Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:06.620" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:36.621" grpc.time_ms=1.36 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2533gitlab # [ 453.830957] gitaly[907]: time="2026-08-28T11:42:36.724Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:46.194" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:36.705" grpc.time_ms=18.851 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2534gitlab # [ 453.904468] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:36.796Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"144b1b541e504e87a5b19299","created_at":"2026-08-28T11:42:36.786Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-4c47ab48dbf64f0a","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:36.793Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-144b1b541e504e87a5b19299: start","job_status":"start","queue_duration_s":0.002841,"scheduling_latency_s":0.002901}2535gitlab # [ 453.949316] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 3062536gitlab # [ 453.955383] nixbot[3630]: {"ts": "2026-08-28T11:42:36.821Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 \"HTTP/1.1 201 Created\""}2537gitlab # [ 454.028836] gitaly[907]: time="2026-08-28T11:42:36.922Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VKRT8N381AHWCE48G9XP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:06.020" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:36.921" grpc.time_ms=1.133 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2538gitlab # [ 454.121886] gitaly[907]: time="2026-08-28T11:42:37.015Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142VKRT8N381AHWCE48G9XP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:46.096" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:36.996" grpc.time_ms=18.39 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2539gitlab # [ 454.166799] gitaly[907]: time="2026-08-28T11:42:37.060Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:37.059" grpc.time_ms=1.041 pid=907 span.kind=server system=grpc2540gitlab # [ 454.217557] gitaly[907]: time="2026-08-28T11:42:37.111Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:37.110" grpc.time_ms=0.704 pid=907 span.kind=server system=grpc2541gitlab # [ 454.244434] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2962542gitlab # [ 454.250425] nixbot[3630]: {"ts": "2026-08-28T11:42:37.124Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 \"HTTP/1.1 201 Created\""}2543gitlab # [ 454.263016] gitaly[907]: time="2026-08-28T11:42:37.156Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-28T11:42:37.155" grpc.time_ms=0.876 pid=907 span.kind=server system=grpc2544gitlab # [ 454.287408] gitaly[907]: time="2026-08-28T11:42:37.163Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:43:07.007" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:37.008" operations="[repack_refs]" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2545gitlab # [ 454.308401] gitaly[907]: time="2026-08-28T11:42:37.168Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX diskcache=e800878e-cfcd-4e7a-bf7c-1364bc219d9f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:43:07.007" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:37.008" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2546gitlab # [ 454.325566] gitaly[907]: time="2026-08-28T11:42:37.168Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:43:07.007" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:37.008" pid=907 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-08-28 11:40:45.32171162 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2547gitlab # [ 454.337805] gitaly[907]: time="2026-08-28T11:42:37.168Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=159 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7061 command.oublock=0 command.real_time_ms=152 command.spawn_token_fork_ms=1 command.system_time_ms=71 command.user_time_ms=88 component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:43:07.007" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:37.008" grpc.time_ms=160.548 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2548gitlab # [ 454.351090] gitaly[907]: time="2026-08-28T11:42:37.208Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:43:07.007" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:37.008" pid=907 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2549gitlab # [ 454.364423] gitaly[907]: time="2026-08-28T11:42:37.208Z" level=info msg="optimized repository" clean_stale_data_time=39.29974ms clean_work_trees_time="163.92µs" commit_graph_time=720ns correlation_id=01M142VKF0GHVEJJAD8HYDN7DX gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.7µs" pid=907 prune_time=540ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="2.1µs" storage_name=default system=housekeeping total_time=39.5062ms2550gitlab # [ 454.370175] gitaly[907]: time="2026-08-28T11:42:37.208Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-28T11:43:07.007" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:37.008" pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2551gitlab # [ 454.379649] gitaly[907]: time="2026-08-28T11:42:37.257Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VM2QTZ9TZ0AWYZY8V9XK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:07.055" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:37.256" grpc.time_ms=1.019 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2552gitlab # [ 454.392643] gitaly[907]: time="2026-08-28T11:42:37.270Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-28T11:43:07.064" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:37.265" grpc.time_ms=4.996 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2553gitlab # [ 454.427259] gitaly[907]: time="2026-08-28T11:42:37.320Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M142VKF0GHVEJJAD8HYDN7DX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-28T11:43:07.119" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-28T11:42:37.319" grpc.time_ms=0.867 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2554gitlab # [ 454.457099] gitaly[907]: time="2026-08-28T11:42:37.325Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M142VM2QTZ9TZ0AWYZY8V9XK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-28T11:42:47.184" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-28T11:42:37.314" grpc.time_ms=10.289 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2555gitlab # [ 454.479804] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.337Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"2c342f71265b3befd7911585","created_at":"2026-08-28T11:42:37.335Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"144b1b541e504e87a5b19299","job_size_bytes":3,"pid":3572,"job_status":"deduplicated","message":"PipelineProcessWorker JID-2c342f71265b3befd7911585: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2556gitlab # [ 454.502806] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.340Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"144b1b541e504e87a5b19299","created_at":"2026-08-28T11:42:36.786Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-4c47ab48dbf64f0a","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:36.793Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3572,"job_status":"rescheduled","message":"PipelineProcessWorker JID-144b1b541e504e87a5b19299: rescheduled"}2557gitlab # [ 454.528096] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.351Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"144b1b541e504e87a5b19299","created_at":"2026-08-28T11:42:36.786Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-4c47ab48dbf64f0a","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:36.793Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-144b1b541e504e87a5b19299: done: 0.555534 sec","job_status":"done","queue_duration_s":0.002841,"scheduling_latency_s":0.002901,"gitaly_calls":3,"gitaly_duration_s":0.176846,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.023458,"redis_read_bytes":239,"redis_write_bytes":21780,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002237,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002217,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.005243,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14465,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.007797,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000581,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000787,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002359,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.096,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.274143,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.45109914,"duration_s":0.555534,"completed_at":"2026-08-28T11:42:37.351Z","load_balancing_strategy":"primary","db_duration_s":0.099602,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2558gitlab # [ 454.589424] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.353Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"39db0b543d8d31bfb64e2806","created_at":"2026-08-28T11:42:36.986Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:36.993Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-39db0b543d8d31bfb64e2806: start","job_status":"start","queue_duration_s":0.360227,"scheduling_latency_s":0.360255}2559gitlab # [ 454.606148] nixbot[3630]: {"ts": "2026-08-28T11:42:37.398Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 \"HTTP/1.1 201 Created\""}2560gitlab # [ 454.608924] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f1bc3ee186b853356efea66d316b0d35f7e7dd26 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2532561gitlab # [ 454.611358] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.423Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"39db0b543d8d31bfb64e2806","created_at":"2026-08-28T11:42:36.986Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:36.993Z","job_size_bytes":23,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-39db0b543d8d31bfb64e2806: done: 0.070128 sec","job_status":"done","queue_duration_s":0.360227,"scheduling_latency_s":0.360255,"redis_calls":7,"redis_duration_s":0.001656,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070128,"completed_at":"2026-08-28T11:42:37.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024409,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2562gitlab # [ 454.655001] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.425Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"581e82012bac490bd92ef69f","created_at":"2026-08-28T11:42:37.181Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:37.185Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-581e82012bac490bd92ef69f: start","job_status":"start","queue_duration_s":0.240346,"scheduling_latency_s":0.240377}2563gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses2564gitlab # [ 454.672080] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.472Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"581e82012bac490bd92ef69f","created_at":"2026-08-28T11:42:37.181Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:37.185Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-581e82012bac490bd92ef69f: done: 0.046804 sec","job_status":"done","queue_duration_s":0.240346,"scheduling_latency_s":0.240377,"redis_calls":7,"redis_duration_s":0.00227,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001634,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046804,"completed_at":"2026-08-28T11:42:37.472Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011477,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2565gitlab # [ 454.717904] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.474Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3519f9797df4b844ed2d32c3","created_at":"2026-08-28T11:42:37.196Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:37.200Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3519f9797df4b844ed2d32c3: start","job_status":"start","queue_duration_s":0.27375,"scheduling_latency_s":0.273777}2566gitlab # [ 454.734945] gitaly[907]: time="2026-08-28T11:42:37.617Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M142VMFAK0YXWY82RXY3ZP37 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-28T11:42:37.608" grpc.time_ms=8.992 pid=907 remote_ip="2001:db8:1::1" span.kind=server system=grpc2567gitlab # [ 454.748994] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/28:11:42:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 712568gitlab # [ 454.751101] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3519f9797df4b844ed2d32c3","created_at":"2026-08-28T11:42:37.196Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:37.200Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3519f9797df4b844ed2d32c3: done: 0.040087 sec","job_status":"done","queue_duration_s":0.27375,"scheduling_latency_s":0.273777,"redis_calls":7,"redis_duration_s":0.005191,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040087,"completed_at":"2026-08-28T11:42:37.514Z","load_balancing_strategy":"primary","db_duration_s":0.004896,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2569gitlab # [ 454.793979] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.516Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"5009a3a31656557f64929a7b","created_at":"2026-08-28T11:42:37.213Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:37.216Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5009a3a31656557f64929a7b: start","job_status":"start","queue_duration_s":0.299689,"scheduling_latency_s":0.299714}2570gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f1bc3ee186b853356efea66d316b0d35f7e7dd26/statuses, in 0.14 seconds)2571(finished: subtest: eval and build post commit statuses, in 9.95 seconds)2572(finished: run the VM test script, in 527.62 seconds)2573gitlab # [ 454.809845] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.581Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"5009a3a31656557f64929a7b","created_at":"2026-08-28T11:42:37.213Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-08-28T11:42:37.216Z","job_size_bytes":3,"pid":3572,"sidekiq_tid":"m8po","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5009a3a31656557f64929a7b: done: 0.065008 sec","job_status":"done","queue_duration_s":0.299689,"scheduling_latency_s":0.299714,"redis_calls":10,"redis_duration_s":0.007125,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.004334,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000575,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065008,"completed_at":"2026-08-28T11:42:37.581Z","load_balancing_strategy":"primary","db_duration_s":0.009491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2574test script finished in 527.65s2575cleanup2576kill QemuMachine (pid 46)2577gitlab # [ 454.857021] sidekiq-cluster[3572]: {"severity":"INFO","time":"2026-08-28T11:42:37.586Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ef056cd360bdf6685435d0b7","created_at":"2026-08-28T11:42:37.217Z","trace_propagation_headers":{"sentry-trace":"b8a1cbb869654841917df9240a879337-b06bf7916c074ea5","baggage":"sentry-trace_id=b8a1cbb869654841917df9240a879337,sentry-release=Unknown"},"correlation_id":"01M142VKF0GHVEJJAD8HYDN7DX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","metaqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2578(finished: cleanup, in 1.68 seconds)2579additionally exposed symbols:2580 gitlab,2581 vlan1,2582 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2583[]2584[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-08-28T11:40:53.237Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2585[]2586[]2587[]2588[]2589[]2590[]2591[]2592[]2593[]2594[]2595[]2596[]2597[]2598[]2599[]2600[]2601[]2602[]2603[]2604[]2605[]2606[]2607[]2608[]2609[]2610[]2611[]2612[]2613[]2614[]2615[]2616[]2617[]2618[]2619[]2620[]2621[]2622[]2623[]2624[]2625[]2626[]2627[]2628[]2629[]2630[]2631[]2632[]2633[]2634[]2635[]2636[]2637[]2638[]2639[]2640[]2641[]2642[]2643[]2644[]2645[]2646[]2647[]2648[]2649[]2650[]2651[]2652[]2653[]2654[]2655[]2656[]2657[]2658[]2659[]2660[]2661[]2662[]2663[]2664[]2665[]2666[]2667[]2668[]2669[]2670[]2671[]2672[]2673[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-08-28T11:42:26.781631Z', 'started_at': '2026-08-28T11:42:27.161243Z', 'finished_at': None, 'eval_queue': None}]2674[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2675[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2676[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2677[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2678[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2679[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2680[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2681[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2682[{'id': 5, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-08-28T11:42:27.113Z', 'started_at': None, 'finished_at': '2026-08-28T11:42:36.769Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'f1bc3ee186b853356efea66d316b0d35f7e7dd26', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-08-28T11:42:37.074Z', 'started_at': None, 'finished_at': '2026-08-28T11:42:37.357Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]