vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #300
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.qibcO4retG', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: e9cd10d7-323e-42fe-a1a7-44b50a7522bf16gitlab # Superblock backups stored on blocks:17(finished: start all VMs, in 73.32 seconds)18gitlab # 32768, 98304, 163840, 22937619gitlab: waiting for unit setup-gitlab.service20gitlab # 21gitlab: waiting for the VM to finish booting22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0m6cbaw7gmbqgz7y1yf0gpx9c5h94m7r-nixos-system-gitlab-test/init regInfo=/nix/store/m51svayh05swqv0aayiw3nzxliy1v7gh-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/m51svayh05swqv0aayiw3nzxliy1v7gh-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000028] arm-pv: using stolen time PV108gitlab # [ 0.000395] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000572] Console: colour dummy device 80x25110gitlab # [ 0.000579] printk: legacy console [tty0] enabled111gitlab # [ 0.000763] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000769] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000841] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000981] landlock: Up and running.115gitlab # [ 0.000983] Yama: becoming mindful.116gitlab # [ 0.001414] LSM support for eBPF active117gitlab # [ 0.001639] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001749] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002977] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003689] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003693] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003846] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004867] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.004960] EFI services will not be available.125gitlab # [ 0.005156] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005785] Detected PIPT I-cache on CPU1127gitlab # [ 0.005884] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.006016] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.006145] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007232] Detected PIPT I-cache on CPU2131gitlab # [ 0.007342] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007476] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007609] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008665] Detected PIPT I-cache on CPU3135gitlab # [ 0.008785] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008918] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.009049] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.009551] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.009576] SMP: Total of 4 processors activated.140gitlab # [ 0.009579] CPU: All CPU(s) started at EL1141gitlab # [ 0.009588] CPU features: detected: Branch Target Identification142gitlab # [ 0.009591] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.009594] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.009597] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.009601] CPU features: detected: Common not Private translations146gitlab # [ 0.009603] CPU features: detected: CRC32 instructions147gitlab # [ 0.009606] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.009609] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.009612] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.009614] CPU features: detected: E0PD151gitlab # [ 0.009616] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.009619] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.009622] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.009624] CPU features: detected: Fine Grained Traps155gitlab # [ 0.009627] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.009631] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.009633] CPU features: detected: LSE atomic instructions158gitlab # [ 0.009636] CPU features: detected: Privileged Access Never159gitlab # [ 0.009638] CPU features: detected: PMUv3160gitlab # [ 0.009640] CPU features: detected: RAS Extension Support161gitlab # [ 0.009643] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.009645] CPU features: detected: Random Number Generator163gitlab # [ 0.009647] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.009650] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.009652] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.009656] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.009785] alternatives: applying system-wide alternatives168gitlab # [ 0.012714] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.012977] Memory: 5963836K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.014491] devtmpfs: initialized171gitlab # [ 0.018382] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.018449] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.018639] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.018643] 0 pages in range for non-PLT usage175gitlab # [ 0.018645] 508288 pages in range for PLT usage176gitlab # [ 0.018731] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.019535] DMI not present or invalid.178gitlab # [ 0.022777] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.025410] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.025690] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.027502] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.027524] audit: initializing netlink subsys (disabled)183gitlab # [ 0.027844] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.028321] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.028324] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.028328] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.028331] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.028333] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.028419] cpuidle: using governor ladder190gitlab # [ 0.028482] cpuidle: using governor menu191gitlab # [ 0.028697] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.028959] ASID allocator initialised with 65536 entries193gitlab # [ 0.030033] Serial: AMBA PL011 UART driver194gitlab # [ 0.035146] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.035441] printk: console [ttyAMA0] enabled196gitlab # [ 0.047350] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.047372] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.047381] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.047390] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.047398] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.047406] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.047414] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.047422] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.059285] fbcon: Taking over console205gitlab # [ 0.059296] ACPI: Interpreter disabled.206gitlab # [ 0.060307] iommu: Default domain type: Translated207gitlab # [ 0.060323] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.070034] SCSI subsystem initialized209gitlab # [ 0.070445] usbcore: registered new interface driver usbfs210gitlab # [ 0.070477] usbcore: registered new interface driver hub211gitlab # [ 0.070495] usbcore: registered new device driver usb212gitlab # [ 0.070774] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.070778] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.070786] PTP clock support registered215gitlab # [ 0.070832] EDAC MC: Ver: 3.0.0216gitlab # [ 0.071319] scmi_core: SCMI protocol bus registered217gitlab # [ 0.072250] FPGA manager framework218gitlab # [ 0.073009] vgaarb: loaded219gitlab # [ 0.075186] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.095239] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.095307] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.096136] netfs: FS-Cache loaded223gitlab # [ 0.096441] pnp: PnP ACPI: disabled224gitlab # [ 0.107951] NET: Registered PF_INET protocol family225gitlab # [ 0.110637] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.172175] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.172251] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.172315] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.172738] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.173614] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.173765] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.173852] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.174027] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.174299] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.174331] NET: Registered PF_XDP protocol family236gitlab # [ 0.174351] PCI: CLS 0 bytes, default 64237gitlab # [ 0.174723] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.187838] kvm [1]: HYP mode not available239gitlab # [ 0.262453] Initialise system trusted keyrings240gitlab # [ 0.262802] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.263569] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.263780] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.279008] Key type asymmetric registered244gitlab # [ 0.279023] Asymmetric key parser 'x509' registered245gitlab # [ 0.279091] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.279381] io scheduler mq-deadline registered247gitlab # [ 0.279386] io scheduler kyber registered248gitlab # [ 0.289681] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.291611] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.292146] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.292164] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.292177] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.292184] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.292228] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.292258] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.292368] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.292378] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.292397] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.292402] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.292406] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.292561] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.293304] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.293634] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.293653] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.293706] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.293725] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.294610] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.294960] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.294977] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.295018] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.295816] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.296109] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.296127] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.296183] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.297058] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.297417] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.297435] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.297475] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.298297] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.298636] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.298653] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.298693] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.299480] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.299784] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.299802] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.299849] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.300625] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.301010] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.301028] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.301075] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.301092] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.301864] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.302167] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.302207] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.303020] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.303393] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.303432] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.304130] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.304478] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.304877] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.305276] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.305293] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.305332] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.306170] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.306504] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.306521] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.306561] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.307555] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.307564] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.307570] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.307633] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.307695] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.307759] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.307822] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.307887] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.307967] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.308032] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.308104] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.308168] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.308270] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.308332] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.308358] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.308384] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.308412] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.308444] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.308470] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.308496] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.308523] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.308550] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.308576] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.308604] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.308642] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.308674] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.308700] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.308727] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.308752] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.308777] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.308803] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.308829] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.308855] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.308881] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.308913] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.308920] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.308925] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.311288] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.328032] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.330981] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.333815] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.336802] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.342631] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.345339] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.348048] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.350834] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.357154] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.359788] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.363053] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.371052] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.373662] msm_serial: driver initialized359gitlab # [ 0.373843] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.373907] STM32 USART driver initialized361gitlab # [ 0.413734] loop: module loaded362gitlab # [ 0.414328] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.419937] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.424548] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.429747] virtio_blk virtio5: [vdb] 12283480 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.435121] megasas: 07.734.00.00-rc1367gitlab # [ 0.437230] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.443047] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.443232] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.448585] Using buffer write method371gitlab # [ 0.448674] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.453320] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.453357] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.458168] Using buffer write method375gitlab # [ 0.458206] Concatenating MTD devices:376gitlab # [ 0.458213] (0): "0.flash"377gitlab # [ 0.458219] (1): "0.flash"378gitlab # [ 0.458225] into device "0.flash"379gitlab # [ 0.612810] Freeing initrd memory: 27184K380gitlab # [ 0.631574] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.640203] thunder_xcv, ver 1.0382gitlab # [ 0.640307] thunder_bgx, ver 1.0383gitlab # [ 0.640370] nicpf, ver 1.0384gitlab # [ 0.642240] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.642255] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.642322] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.642337] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.642459] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.642469] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.642557] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.642568] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.642985] sky2: driver version 1.30393gitlab # [ 0.647978] usbcore: registered new interface driver usb-storage394gitlab # [ 0.648125] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.648172] usbserial: USB Serial support registered for generic396gitlab # [ 0.648397] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.648461] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.649175] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.650116] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.652965] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.653022] rtc-pl031 9010000.pl031: setting system clock to 2026-09-03T10:13:03 UTC (1788430383)402gitlab # [ 0.653955] i2c_dev: i2c /dev entries driver403gitlab # [ 0.659275] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00404gitlab # [ 0.660700] hub 1-0:1.0: USB hub found405gitlab # [ 0.660912] hub 1-0:1.0: 6 ports detected406gitlab # [ 0.662276] sdhci: Secure Digital Host Controller Interface driver407gitlab # [ 0.662288] sdhci: Copyright(c) Pierre Ossman408gitlab # [ 0.663023] Synopsys Designware Multimedia Card Interface Driver409gitlab # [ 0.664099] sdhci-pltfm: SDHCI platform and OF driver helper410gitlab # [ 0.668348] hid: raw HID events driver (C) Jiri Kosina411gitlab # [ 0.669132] usbcore: registered new interface driver usbhid412gitlab # [ 0.669140] usbhid: USB HID core driver413gitlab # [ 0.677584] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.681469] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.681729] NET: Registered PF_INET6 protocol family416gitlab # [ 0.684248] Segment Routing with IPv6417gitlab # [ 0.684287] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.684518] NET: Registered PF_PACKET protocol family419gitlab # [ 0.684929] 9pnet: Installing 9P2000 support420gitlab # [ 0.690027] Key type dns_resolver registered421gitlab # [ 0.708539] registered taskstats version 1422gitlab # [ 0.709265] Loading compiled-in X.509 certificates423gitlab # [ 0.735422] Demotion targets for Node 0: null424gitlab # [ 0.735676] Key type .fscrypt registered425gitlab # [ 0.735684] Key type fscrypt-provisioning registered426gitlab # [ 0.735933] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.735958] ima: Allocated hash algorithm: sha1428gitlab # [ 0.736020] ima: No architecture policies found429gitlab # [ 0.737437] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.767878] clk: Disabling unused clocks431gitlab # [ 0.767918] PM: genpd: Disabling unused power domains432gitlab # [ 0.773688] Freeing unused kernel memory: 4736K433gitlab # [ 0.773939] Run /init as init process434gitlab # [ 0.821280] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.911255] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.063546] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.151854] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0438gitlab # [ 1.156103] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)439gitlab # [ 1.156147] systemd[1]: Detected virtualization qemu.440gitlab # [ 1.156205] systemd[1]: Detected architecture arm64.441gitlab # [ 1.156220] systemd[1]: Running in initrd.442gitlab # [ 1.157068] systemd[1]: Initializing machine ID from random generator.443gitlab # [ 1.157322] systemd[1]: Hostname set to <gitlab>.444gitlab # [ 1.271260] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.417089] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.423323] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2447gitlab # [ 1.423789] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0448gitlab # [ 1.518167] systemd[1]: Queued start job for default target Initrd Default Target.449gitlab # [ 1.534785] systemd[1]: Created slice Slice /system/modprobe.450gitlab # [ 1.535107] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.451gitlab # [ 1.535151] systemd[1]: Expecting device /dev/disk/by-label/nix-store...452gitlab # [ 1.535169] systemd[1]: Expecting device /dev/disk/by-label/nixos...453gitlab # [ 1.535372] systemd[1]: Reached target Path Units.454gitlab # [ 1.535397] systemd[1]: Reached target Slice Units.455gitlab # [ 1.535421] systemd[1]: Reached target Swaps.456gitlab # [ 1.535446] systemd[1]: Reached target Timer Units.457gitlab # [ 1.535667] systemd[1]: Listening on D-Bus System Message Bus Socket.458gitlab # [ 1.535932] systemd[1]: Listening on Journal Socket (/dev/log).459gitlab # [ 1.536130] systemd[1]: Listening on Journal Sockets.460gitlab # [ 1.536318] systemd[1]: Listening on udev Control Socket.461gitlab # [ 1.536488] systemd[1]: Listening on udev Kernel Socket.462gitlab # [ 1.536517] systemd[1]: Reached target Socket Units.463gitlab # [ 1.538808] systemd[1]: Starting Create List of Static Device Nodes...464gitlab # [ 1.541138] systemd[1]: Starting Load Kernel Module 9pnet_virtio...465gitlab # [ 1.541266] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs466gitlab # [ 1.551479] systemd[1]: Mounting Kernel Configuration File System...467gitlab # [ 1.578084] systemd[1]: Starting Journal Service...468gitlab # [ 1.582019] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.582264] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.585739] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.593209] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.598270] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.607665] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.608561] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.617328] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.476gitlab # [ 1.617784] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...477gitlab # [ 1.617938] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev478gitlab # [ 1.623555] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0479gitlab # [ 1.623728] [drm] features: -virgl +edid -resource_blob -host_visible480gitlab # [ 1.623739] [drm] features: -context_init481gitlab # [ 1.624682] [drm] number of scanouts: 1482gitlab # [ 1.624707] [drm] number of cap sets: 0483gitlab # [ 1.629450] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic484gitlab # [ 1.629469] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0485gitlab # [ 1.634040] systemd-journald[96]: Collecting audit messages is disabled.486gitlab # [ 1.634240] Console: switching to colour frame buffer device 160x50487gitlab # [ 1.640871] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device488gitlab # [ 1.655798] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.489gitlab # [ 1.659397] systemd[1]: Starting Create Static Device Nodes in /dev...490gitlab # [ 1.664789] systemd[1]: Finished Load Kernel Modules.491gitlab # [ 1.666986] systemd[1]: Starting Apply Kernel Variables...492gitlab # [ 1.704516] systemd[1]: Finished Apply Kernel Variables.493gitlab # [ 1.714527] systemd[1]: Started Journal Service.494gitlab # [ 1.711392] systemd-modules-load[97]: Using 4 probe threads495gitlab # [ 1.715251] systemd-modules-load[97]: Module 'virtio_balloon' is built in496gitlab # [ 1.716457] systemd-modules-load[97]: Module 'virtio_console' is built in497gitlab # [ 1.717600] systemd-modules-load[97]: Module 'virtio_rng' is built in498gitlab # [ 1.718672] systemd-modules-load[97]: Inserted module 'dm_mod'499gitlab # [ 1.719675] systemd-modules-load[97]: Inserted module 'virtio_gpu'500gitlab # [ 1.720759] systemd[1]: Finished Create Static Device Nodes in /dev.501gitlab # [ 1.721848] systemd[1]: Reached target Preparation for Local File Systems.502gitlab # [ 1.722949] systemd[1]: Reached target Local File Systems.503gitlab # [ 1.724277] systemd[1]: Starting Create System Files and Directories...504gitlab # [ 1.725603] systemd[1]: Starting Rule-based Manager for Device Events and Files...505gitlab # [ 1.748662] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.760856] systemd-udevd[109]: Using default interface naming scheme 'v261'.507gitlab # [ 1.776345] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.798428] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.866201] systemd-vconsole-setup[129]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.870724] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.223747] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.226542] systemd[1]: Reached target System Initialization.513gitlab # [ 2.229225] systemd[1]: Reached target Basic System.514gitlab # [ 2.430516] (udev-worker)[138]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.435836] (udev-worker)[138]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.438968] (udev-worker)[119]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.442069] systemd[1]: Found device /dev/disk/by-label/nix-store.518gitlab # [ 2.454325] systemd[1]: Found device /dev/disk/by-label/nixos.519gitlab # [ 2.463885] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.467128] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.514387] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.519034] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.587848] systemd[1]: Mounting /sysroot...524gitlab # [ 2.621578] EXT4-fs (vda): mounted filesystem e9cd10d7-323e-42fe-a1a7-44b50a7522bf r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.621398] systemd[1]: Mounted /sysroot.526gitlab # [ 2.623461] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.627585] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.629042] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.631846] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.648687] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.655425] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.658720] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.662871] systemd[1]: Mounted /sysroot/nix/.rw-store.534gitlab # [ 2.664271] systemd[1]: Mounted /sysroot/run.535gitlab # [ 2.669105] systemd-sysroot-fstab-check[164]: /sysroot should be mounted in the initrd, will request daemon-reload.536gitlab # [ 2.673243] systemd[1]: Starting rw-sysroot-nix-store.service...537gitlab # [ 2.674684] systemd[1]: Reload requested from client PID 164 ('systemd-sysroot') (unit initrd-parse-etc.service)...538gitlab # [ 2.676484] systemd[1]: Reloading...539gitlab # [ 2.691238] erofs (device vdb): mounted with root inode @ nid 128.540gitlab # [ 2.778707] systemd[1]: Reloading finished in 101 ms.541gitlab # [ 2.832095] systemd-sysroot-fstab-check[164]: Requesting initrd-fs.target/start/replace...542gitlab # [ 2.833362] systemd[1]: Mounted /sysroot/nix/.ro-store.543gitlab # [ 2.834167] systemd[1]: Mounted /sysroot/tmp/shared.544gitlab # [ 2.834916] systemd[1]: Mounted /sysroot/tmp/xchg.545gitlab # [ 2.835660] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.546gitlab # [ 2.836842] systemd[1]: Finished rw-sysroot-nix-store.service.547gitlab # [ 2.839044] systemd-sysroot-fstab-check[164]: Requesting swap.target/start/replace...548gitlab # [ 2.843624] systemd[1]: initrd-parse-etc.service: Deactivated successfully.549gitlab # [ 2.844955] systemd[1]: Finished Mountpoints Configured in the Real Root.550gitlab # [ 2.846473] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.551gitlab # [ 2.847555] systemd[1]: Starting rw-sysroot-nix-store.service...552gitlab # [ 2.872778] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.553gitlab # [ 2.875551] systemd[1]: Finished rw-sysroot-nix-store.service.554gitlab # [ 2.877478] systemd[1]: Mounting /sysroot/nix/store...555gitlab # [ 2.936557] systemd[1]: Mounted /sysroot/nix/store.556gitlab # [ 2.938638] systemd[1]: Reached target Initrd File Systems.557gitlab # [ 2.939550] systemd[1]: Starting Find NixOS closure...558gitlab # [ 2.944621] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...559gitlab # [ 2.965230] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.560gitlab # [ 2.976168] systemd[1]: Finished Find NixOS closure.561gitlab # [ 2.980075] systemd[1]: Reached target Initrd Default Target.562gitlab # [ 2.981192] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...563gitlab # [ 2.995546] systemd[1]: Stopped target Initrd Default Target.564gitlab # [ 2.996547] systemd[1]: Stopped target Basic System.565gitlab # [ 2.997298] systemd[1]: Stopped target Initrd Root Device.566gitlab # [ 2.998117] systemd[1]: Stopped target Path Units.567gitlab # [ 2.998847] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.568gitlab # [ 2.999925] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.569gitlab # [ 3.001118] systemd[1]: Stopped target Slice Units.570gitlab # [ 3.001886] systemd[1]: Stopped target Socket Units.571gitlab # [ 3.002758] systemd[1]: Stopped target System Initialization.572gitlab # [ 3.003621] systemd[1]: Stopped target Swaps.573gitlab # [ 3.004316] systemd[1]: Stopped target Timer Units.574gitlab # [ 3.005050] systemd[1]: dbus.socket: Deactivated successfully.575gitlab # [ 3.005902] systemd[1]: Closed D-Bus System Message Bus Socket.576gitlab # [ 3.007016] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.577gitlab # [ 3.008927] systemd[1]: Stopped Find NixOS closure.578gitlab # [ 3.010330] systemd[1]: Starting Load Kernel Module 9pnet_virtio...579gitlab # [ 3.011625] systemd[1]: Starting rw-sysroot-nix-store.service...580gitlab # [ 3.013395] systemd[1]: systemd-sysctl.service: Deactivated successfully.581gitlab # [ 3.014412] systemd[1]: Stopped Apply Kernel Variables.582gitlab # [ 3.015508] systemd[1]: systemd-modules-load.service: Deactivated successfully.583gitlab # [ 3.017215] systemd[1]: Stopped Load Kernel Modules.584gitlab # [ 3.018925] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.585gitlab # [ 3.020192] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.586gitlab # [ 3.021342] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.587gitlab # [ 3.022416] systemd[1]: Stopped Create System Files and Directories.588gitlab # [ 3.023334] systemd[1]: Stopped target Local File Systems.589gitlab # [ 3.024298] systemd[1]: Stopped target Preparation for Local File Systems.590gitlab # [ 3.025277] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.591gitlab # [ 3.026313] systemd[1]: Stopped Coldplug All udev Devices.592gitlab # [ 3.027111] systemd[1]: Stopping Rule-based Manager for Device Events and Files...593gitlab # [ 3.028249] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.594gitlab # [ 3.029296] systemd[1]: Stopped Virtual Console Setup.595gitlab # [ 3.030046] systemd[1]: initrd-cleanup.service: Deactivated successfully.596gitlab # [ 3.030993] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.597gitlab # [ 3.031901] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.598gitlab # [ 3.032997] systemd[1]: Finished Load Kernel Module 9pnet_virtio.599gitlab # [ 3.033848] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.600gitlab # [ 3.034834] systemd[1]: Finished rw-sysroot-nix-store.service.601gitlab # [ 3.054461] systemd[1]: systemd-udevd.service: Deactivated successfully.602gitlab # [ 3.055451] systemd[1]: Stopped Rule-based Manager for Device Events and Files.603gitlab # [ 3.056543] systemd[1]: systemd-udevd.service: Consumed 2.017s CPU time over 1.333s wall clock time, 27.8M memory peak.604gitlab # [ 3.058023] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.605gitlab # [ 3.059043] systemd[1]: Closed udev Control Socket.606gitlab # [ 3.059764] systemd[1]: Starting Cleanup udev Database...607gitlab # [ 3.061768] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.608gitlab # [ 3.062863] systemd[1]: Stopped Create Static Device Nodes in /dev.609gitlab # [ 3.063759] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.610gitlab # [ 3.065005] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.611gitlab # [ 3.066029] systemd[1]: kmod-static-nodes.service: Deactivated successfully.612gitlab # [ 3.067022] systemd[1]: Stopped Create List of Static Device Nodes.613gitlab # [ 3.118634] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.614gitlab # [ 3.121711] systemd[1]: Finished Cleanup udev Database.615gitlab # [ 3.123248] systemd[1]: Reached target Switch Root.616gitlab # [ 3.124799] systemd[1]: Starting NixOS Activation...617gitlab # [ 3.288729] initrd-nixos-activation-start[218]: booting system configuration /nix/store/0m6cbaw7gmbqgz7y1yf0gpx9c5h94m7r-nixos-system-gitlab-test618gitlab # [ 3.350877] initrd-nixos-activation-start[218]: running activation script...619gitlab # [ 3.679637] initrd-nixos-activation-start[241]: setting up /etc...620gitlab # [ 3.842062] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.621gitlab # [ 3.845645] systemd[1]: Finished NixOS Activation.622gitlab # [ 3.847831] systemd[1]: Starting Switch Root...623gitlab # [ 3.903135] systemd[1]: Switching root.624gitlab # [ 4.011669] systemd-journald[96]: Received SIGTERM from PID 1 (systemd).625gitlab # [ 4.660904] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)626gitlab # [ 4.661081] systemd[1]: Detected virtualization qemu.627gitlab # [ 4.661196] systemd[1]: Detected architecture arm64.628gitlab # [ 4.661400] systemd[1]: Detected first boot.629gitlab # [ 4.662855] systemd[1]: Initializing machine ID from random generator.630gitlab # [ 4.946149] systemd[1]: bpf-restrict-fs: LSM BPF program attached631gitlab # [ 5.072302] systemd[1]: Applying preset policy.632gitlab # [ 5.271376] systemd[1]: Populated /etc with preset unit settings.633gitlab # [ 5.550170] systemd[1]: initrd-switch-root.service: Deactivated successfully.634gitlab # [ 5.550605] systemd[1]: Stopped initrd-switch-root.service.635gitlab # [ 5.557330] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.636gitlab # [ 5.563239] systemd[1]: Created slice Slice /system/getty.637gitlab # [ 5.569341] systemd[1]: Created slice GitLab DevOps Platform Slice.638gitlab # [ 5.574302] systemd[1]: Created slice User and Session Slice.639gitlab # [ 5.576725] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.640gitlab # [ 5.578145] systemd[1]: Started Forward Password Requests to Wall Directory Watch.641gitlab # [ 5.579396] systemd[1]: Expecting device /dev/hvc0...642gitlab # [ 5.580576] systemd[1]: Expecting device /dev/ttyAMA0...643gitlab # [ 5.581895] systemd[1]: Reached target Local Encrypted Volumes.644gitlab # [ 5.583384] systemd[1]: Stopped target initrd-fs.target.645gitlab # [ 5.584491] systemd[1]: Stopped target initrd-root-fs.target.646gitlab # [ 5.585639] systemd[1]: Stopped target initrd-switch-root.target.647gitlab # [ 5.587163] systemd[1]: Reached target Virtual Machines and Containers.648gitlab # [ 5.588820] systemd[1]: Reached target Path Units.649gitlab # [ 5.590217] systemd[1]: Reached target Remote File Systems.650gitlab # [ 5.592001] systemd[1]: Reached target Slice Units.651gitlab # [ 5.593829] systemd[1]: Reached target Swaps.652gitlab # [ 5.602491] systemd[1]: Listening on Query the User Interactively for a Password.653gitlab # [ 5.610491] systemd[1]: Listening on Process Core Dump Socket.654gitlab # [ 5.616179] systemd[1]: Listening on Credential Encryption/Decryption.655gitlab # [ 5.621928] systemd[1]: Listening on Factory Reset Management.656gitlab # [ 5.624087] systemd[1]: Listening on Hostname Service Socket.657gitlab # [ 5.635067] systemd[1]: Starting Journal Log Access Socket...658gitlab # [ 5.638067] systemd[1]: Listening on Journal Audit Socket.659gitlab # [ 5.642954] systemd[1]: Listening on Console Output Muting Service Socket.660gitlab # [ 5.644850] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.661gitlab # [ 5.645898] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os662gitlab # [ 5.646961] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki663gitlab # [ 5.659597] systemd[1]: Listening on Disk Repartitioning Service Socket.664gitlab # [ 5.661729] systemd[1]: Listening on udev Control Socket.665gitlab # [ 5.664274] systemd[1]: Listening on udev Varlink Socket.666gitlab # [ 5.673954] systemd[1]: Mounting Huge Pages File System...667gitlab # [ 5.682201] systemd[1]: Mounting POSIX Message Queue File System...668gitlab # [ 5.694215] systemd[1]: Mounting Kernel Debug File System...669gitlab # [ 5.712262] systemd[1]: Mounting Kernel Trace File System...670gitlab # [ 5.727179] systemd[1]: Starting Create List of Static Device Nodes...671gitlab # [ 5.742792] systemd[1]: Starting Load Kernel Module 9pnet_virtio...672gitlab # [ 5.747572] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs673gitlab # [ 5.759653] systemd[1]: Mounting Kernel Configuration File System...674gitlab # [ 5.761044] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm675gitlab # [ 5.762418] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore676gitlab # [ 5.776845] systemd[1]: Starting Load Kernel Module fuse...677gitlab # [ 5.777786] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67678gitlab # [ 5.788333] systemd[1]: Starting Journal Service...679gitlab # [ 5.793924] systemd[1]: Starting Load Kernel Modules...680gitlab # [ 5.803887] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...681gitlab # [ 5.819235] systemd[1]: Starting Remount Root and Kernel File Systems...682gitlab # [ 5.822694] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os683gitlab # [ 5.835454] systemd[1]: Starting Coldplug All udev Devices...684gitlab # [ 5.847729] systemd[1]: Listening on Journal Log Access Socket.685gitlab # [ 5.848441] systemd[1]: Mounted Huge Pages File System.686gitlab # [ 5.849417] systemd[1]: Mounted POSIX Message Queue File System.687gitlab # [ 5.850379] systemd[1]: Mounted Kernel Debug File System.688gitlab # [ 5.851448] systemd[1]: Mounted Kernel Trace File System.689gitlab # [ 5.852580] systemd[1]: Finished Create List of Static Device Nodes.690gitlab # [ 5.853747] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.691gitlab # [ 5.854630] systemd[1]: Finished Load Kernel Module 9pnet_virtio.692gitlab # [ 5.855577] systemd[1]: Mounted Kernel Configuration File System.693gitlab # [ 5.868037] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...694gitlab # [ 5.878435] fuse: init (API version 7.45)695gitlab # [ 5.880886] systemd[1]: modprobe@fuse.service: Deactivated successfully.696gitlab # [ 5.887451] EXT4-fs (vda): re-mounted e9cd10d7-323e-42fe-a1a7-44b50a7522bf.697gitlab # [ 5.887678] systemd[1]: Finished Load Kernel Module fuse.698gitlab # [ 5.891819] systemd[1]: Finished Load Kernel Modules.699gitlab # [ 5.896685] systemd-journald[314]: Collecting audit messages is enabled.700gitlab # [ 5.900778] systemd[1]: Finished Remount Root and Kernel File Systems.701gitlab # [ 5.902897] systemd[1]: Listening on Disk Image Download Service Socket.702gitlab # [ 5.908826] systemd[1]: Queued start job for default target Multi-User System.703gitlab # [ 5.910923] systemd[1]: Unnecessary job was removed for /dev/vdb.704gitlab # [ 5.914488] systemd[1]: systemd-journald.service: Deactivated successfully.705gitlab # [ 5.918019] systemd-modules-load[315]: Using 3 probe threads[ 5.922120] systemd[1]: Mounting FUSE Control File System...706gitlab # 707gitlab # [ 5.923581] systemd-modules-load[315]: Module 'atkbd' is built in708gitlab # [ 5.929408] systemd[1]: Starting Firewall...709gitlab # [ 5.930104] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore710gitlab # [ 5.926223] systemd-modules-load[315]: Module 'loop' is built in[ 5.937538] systemd[1]: Starting Load/Save OS Random Seed...711gitlab # 712gitlab # [ 5.941257] systemd[1]: Starting Apply Kernel Variables...713gitlab # [ 5.942646] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os714gitlab # [ 5.947325] systemd[1]: Started Journal Service.715gitlab # [ 5.947210] systemd-modules-load[315]: Inserted module 'tls'716gitlab # [ 5.951672] systemd-oomd[316]: No swap; memory pressure usage will be degraded717gitlab # [ 5.956249] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.718gitlab # [ 5.960309] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.719gitlab # [ 5.961403] systemd[1]: Mounted FUSE Control File System.720gitlab # [ 5.989401] systemd[1]: Starting Flush Journal to Persistent Storage...721gitlab # [ 5.993458] systemd[1]: Starting Create Static Device Nodes in /dev...722gitlab # [ 5.996236] systemd[1]: Finished Load/Save OS Random Seed.723gitlab # [ 5.998602] systemd[1]: Finished Apply Kernel Variables.724gitlab # [ 6.002207] systemd[1]: Reached target First Boot Complete.725gitlab # [ 6.022185] systemd-journald[314]: Received client request to flush runtime journal.726gitlab # [ 6.040715] systemd[1]: Finished Create Static Device Nodes in /dev.727gitlab # [ 6.042779] systemd[1]: Reached target Preparation for Local File Systems.728gitlab # [ 6.043885] systemd[1]: Starting Rule-based Manager for Device Events and Files...729gitlab # [ 6.045344] systemd[1]: Finished Flush Journal to Persistent Storage.730gitlab # [ 6.120833] systemd-udevd[355]: Using default interface naming scheme 'v261'.731gitlab # [ 6.182232] systemd[1]: Started Rule-based Manager for Device Events and Files.732gitlab # [ 6.390460] systemd[1]: Starting Virtual Console Setup...733gitlab # [ 6.467588] systemd-vconsole-setup[439]: Configuration of first virtual console was skipped, ignoring remaining ones.734gitlab # [ 6.473403] systemd[1]: Finished Virtual Console Setup.735gitlab # [ 6.541522] systemd[1]: Finished Firewall.736gitlab # [ 6.551152] systemd[1]: Mounting /run/wrappers...737gitlab # [ 6.586980] systemd[1]: Mounted /run/wrappers.738gitlab # [ 6.589440] systemd[1]: Reached target Local File Systems.739gitlab # [ 6.592423] systemd[1]: Listening on Boot Loader Control Service Socket.740gitlab # [ 6.597367] systemd[1]: Starting register-nix-paths.service...741gitlab # [ 6.601758] systemd[1]: Starting Create SUID/SGID Wrappers...742gitlab # [ 6.604845] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.743gitlab # [ 6.628191] systemd[1]: Starting Save Transient machine-id to Disk...744gitlab # [ 6.636898] systemd[1]: Starting Create System Files and Directories...745gitlab # [ 6.681255] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.746gitlab # [ 6.685655] systemd[1]: Finished Save Transient machine-id to Disk.747gitlab # [ 6.712154] systemd[1]: Finished Create System Files and Directories.748gitlab # [ 6.719448] systemd[1]: Starting Rebuild Journal Catalog...749gitlab # [ 6.724303] systemd[1]: Starting Record System Boot/Shutdown in UTMP...750gitlab # [ 6.767501] systemd[1]: Finished Record System Boot/Shutdown in UTMP.751gitlab # [ 6.787338] systemd[1]: Finished Rebuild Journal Catalog.752gitlab # [ 6.796375] systemd[1]: Starting Update is Completed...753gitlab # [ 6.834616] systemd[1]: Finished Update is Completed.754gitlab # [ 6.901023] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.755gitlab # [ 6.904313] systemd[1]: Finished Create SUID/SGID Wrappers.756gitlab # [ 6.905502] systemd[1]: suid-sgid-wrappers.service: Consumed 217ms CPU time over 300ms wall clock time, 7.1M memory peak, 1.9M read from disk.757gitlab # [ 6.930251] systemd[1]: Finished Coldplug All udev Devices.758gitlab # [ 6.995609] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs759gitlab # [ 7.100840] (udev-worker)[402]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.760gitlab # [ 7.106622] (udev-worker)[402]: Network interface NamePolicy= disabled on kernel command line.761gitlab # [ 7.117864] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.762gitlab # [ 7.135657] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.763gitlab # [ 7.150645] (udev-worker)[409]: Network interface NamePolicy= disabled on kernel command line.764gitlab # [ 7.165975] systemd[1]: Condition check resulted in Virtio network device being skipped.765gitlab # [ 7.167836] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore766gitlab # [ 7.176196] systemd[1]: Starting Create SUID/SGID Wrappers...767gitlab # [ 7.177545] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.768gitlab # [ 7.179313] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67769gitlab # [ 7.185488] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore770gitlab # [ 7.189115] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os771gitlab # [ 7.192132] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os772gitlab # [ 7.305116] systemd[1]: Finished register-nix-paths.service.773gitlab # [ 7.309525] mousedev: PS/2 mouse device common for all mice774gitlab # [ 7.362375] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.775gitlab # [ 7.363536] systemd[1]: Finished Create SUID/SGID Wrappers.776gitlab # [ 7.364401] systemd[1]: Reached target System Initialization.777gitlab # [ 7.365507] systemd[1]: Started Discard unused filesystem blocks once a week.778gitlab # [ 7.367067] systemd[1]: Started logrotate.timer.779gitlab # [ 7.367790] systemd[1]: Started Daily Cleanup of Temporary Directories.780gitlab # [ 7.368795] systemd[1]: Reached target Timer Units.781gitlab # [ 7.369543] systemd[1]: Listening on D-Bus System Message Bus Socket.782gitlab # [ 7.370462] systemd[1]: Listening on Nix Daemon Socket.783gitlab # [ 7.373651] systemd[1]: Starting nixbot CI service socket...784gitlab # [ 7.375037] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.785gitlab # [ 7.376394] systemd[1]: Listening on nixbot CI service socket.786gitlab # [ 7.377231] systemd[1]: Reached target Socket Units.787gitlab # [ 7.379132] systemd[1]: Starting D-Bus System Message Bus...788gitlab # [ 7.413854] dbus-broker-launch[623]: Looking up NSS user entry for 'systemd-timesync'...789gitlab # [ 7.416002] dbus-broker-launch[623]: NSS returned no entry for 'systemd-timesync'790gitlab # [ 7.417671] dbus-broker-launch[623]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"791gitlab # [ 7.420222] systemd[1]: Started D-Bus System Message Bus.792gitlab # [ 7.431797] dbus-broker-launch[623]: Ready793gitlab # [ 7.434107] systemd[1]: Reached target Basic System.794gitlab # [ 7.436050] systemd[1]: Started backdoor.service.795gitlab # [ 7.438489] systemd[1]: Starting gitlab-config.service...796gitlab # [ 7.440780] systemd[1]: Starting Import lastlog data into lastlog2 database...797gitlab # [ 7.456686] systemd[1]: Starting Logrotate configuration check...798gitlab # [ 7.470573] systemd[1]: Starting Name Service Cache Daemon (nsncd)...799gitlab # [ 7.475281] systemd[1]: Starting Post-Boot Actions...800gitlab # connecting to host...801gitlab # [ 7.479440] logrotate[628]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.802gitlab # [ 7.486441] logrotate[628]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf803gitlab # [ 7.488267] logrotate[628]: note: 'monthly' overrides previously specified 'weekly'804gitlab # [ 7.489560] logrotate[628]: Reading state from file: /var/lib/logrotate.status805gitlab # [ 7.490804] logrotate[628]: state file /var/lib/logrotate.status does not exist806gitlab # [ 7.496206] logrotate[628]: Allocating hash table for state file, size 64 entries807gitlab # [ 7.497348] logrotate[628]: Handling 4 logs808gitlab # [ 7.497944] logrotate[628]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed809gitlab: Guest shell says: b'Spawning backdoor root shell...\n'810gitlab # [ 7.501597] logrotate[628]: considering log /var/log/btmp811gitlab # [ 7.502423] logrotate[628]: Creating new state812gitlab: connected to guest root shell813gitlab # [ 7.503050] logrotate[628]: Now: 2026-09-03 10:13814gitlab: (connecting took 7.97 seconds)815gitlab # [ 7.503692] logrotate[628]: Last rotated at 2026-09-03 10:00816gitlab: (finished: waiting for the VM to finish booting, in 7.97 seconds)817gitlab # [ 7.509829] logrotate[628]: log does not need rotating (log has already been rotated)818gitlab # [ 7.510889] logrotate[628]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed819gitlab # [ 7.512831] logrotate[628]: considering log /var/log/wtmp820gitlab # [ 7.513507] logrotate[628]: Creating new state821gitlab # [ 7.514071] logrotate[628]: Now: 2026-09-03 10:13822gitlab # [ 7.514685] logrotate[628]: Last rotated at 2026-09-03 10:00823gitlab # [ 7.515393] logrotate[628]: log does not need rotating (log has already been rotated)824gitlab # [ 7.517056] logrotate[628]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed825gitlab # [ 7.519020] logrotate[628]: switching euid from 0 to 165 and egid from 0 to 165 (pid 628)826gitlab # [ 7.520717] logrotate[628]: considering log /var/gitlab/state/log/*.log827gitlab # [ 7.521689] logrotate[628]: log /var/gitlab/state/log/*.log does not exist -- skipping828gitlab # [ 7.522962] logrotate[628]: Creating new state829gitlab # [ 7.523593] logrotate[628]: switching euid from 165 to 0 and egid from 165 to 0 (pid 628)830gitlab # [ 7.524850] logrotate[628]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed831gitlab # [ 7.526549] logrotate[628]: switching euid from 0 to 60 and egid from 0 to 60 (pid 628)832gitlab # [ 7.527575] logrotate[628]: considering log /var/log/nginx/*.log833gitlab # [ 7.529483] logrotate[628]: log /var/log/nginx/*.log does not exist -- skipping834gitlab # [ 7.535154] logrotate[628]: Creating new state835gitlab # [ 7.535852] logrotate[628]: not running postrotate script, since no logs were rotated836gitlab # [ 7.539065] logrotate[628]: switching euid from 60 to 0 and egid from 60 to 0 (pid 628)837gitlab # [ 7.540326] systemd[1]: Started Reset console on configuration changes.838gitlab # [ 7.541642] nsncd[629]: Sep 03 10:13:10.356 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"839gitlab # [ 7.543988] systemd[1]: Starting resolvconf update...840gitlab # [ 7.545137] systemd[1]: Started Name Service Cache Daemon (nsncd).841gitlab # [ 7.546331] systemd[1]: Finished Logrotate configuration check.842gitlab # [ 7.547253] systemd[1]: Finished Post-Boot Actions.843gitlab # [ 7.548113] systemd[1]: Reached target Host and Network Name Lookups.844gitlab # [ 7.549142] systemd[1]: Reached target User and Group Name Lookups.845gitlab # [ 7.550239] systemd[1]: Starting User Login Management...846gitlab # [ 7.551045] systemd[1]: Finished Import lastlog data into lastlog2 database.847gitlab # [ 7.605718] systemd-logind[659]: New seat seat0.848gitlab # [ 7.609187] systemd[1]: Stopped target Host and Network Name Lookups.849gitlab # [ 7.610275] systemd[1]: Stopping Host and Network Name Lookups...850gitlab # [ 7.611092] systemd[1]: Stopped target User and Group Name Lookups.851gitlab # [ 7.617848] systemd[1]: Stopping User and Group Name Lookups...852gitlab # [ 7.618703] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...853gitlab # [ 7.619620] systemd-logind[659]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)854gitlab # [ 7.621746] systemd-logind[659]: Watching system buttons on /dev/input/event0 (gpio-keys)855gitlab # [ 7.622978] systemd[1]: Started User Login Management.856gitlab # [ 7.623725] systemd[1]: nscd.service: Deactivated successfully.857gitlab # [ 7.624603] systemd[1]: Stopped Name Service Cache Daemon (nsncd).858gitlab # [ 7.625781] systemd[1]: nscd.service: Consumed 25ms CPU time over 142ms wall clock time, 7.7M memory peak, 5.8M read from disk.859gitlab # [ 7.627875] systemd[1]: Starting linger-users.service...860gitlab # [ 7.633793] systemd[1]: Starting Name Service Cache Daemon (nsncd)...861gitlab # [ 7.655407] nsncd[723]: Sep 03 10:13:10.505 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"862gitlab # [ 7.658927] systemd[1]: Started Name Service Cache Daemon (nsncd).863gitlab # [ 7.659860] systemd[1]: Reached target Host and Network Name Lookups.864gitlab # [ 7.660871] systemd[1]: Reached target User and Group Name Lookups.865gitlab # [ 7.661793] systemd[1]: linger-users.service: Deactivated successfully.866gitlab # [ 7.662865] systemd[1]: Finished linger-users.service.867gitlab # [ 7.668987] systemd[1]: Finished resolvconf update.868gitlab # [ 7.671375] systemd[1]: Reached target Preparation for Network.869gitlab # [ 7.677395] systemd[1]: Starting DHCP Client...870gitlab # [ 7.680710] systemd[1]: Starting Address configuration of eth1...871gitlab # [ 7.688700] systemd[1]: Starting Extra networking commands....872gitlab # [ 7.721855] network-addresses-eth1-start[746]: adding address 192.168.1.1/24... done873gitlab # [ 7.726433] network-addresses-eth1-start[746]: adding address 2001:db8:1::1/64... done874gitlab # [ 7.733598] systemd[1]: Finished Address configuration of eth1.875gitlab # [ 7.812427] dhcpcd[761]: dhcpcd-10.3.2 starting876gitlab # [ 7.814155] systemd[1]: Finished Extra networking commands..877gitlab # [ 7.816495] dhcpcd[812]: dev: loaded udev878gitlab # [ 7.818157] systemd[1]: Reached target Network.879gitlab # [ 7.827227] systemd[1]: Starting gitlab-workhorse.service...880gitlab # [ 7.829688] systemd[1]: Starting Nginx Web Server...881gitlab # [ 7.834697] systemd[1]: Starting PostgreSQL Server...882gitlab # [ 7.839237] 8021q: 802.1Q VLAN Support v1.8883gitlab # [ 7.839679] 8021q: adding VLAN 0 to HW filter on device eth1884gitlab # [ 7.840770] systemd[1]: Starting Redis Server - redis-gitlab...885gitlab # [ 7.848739] systemd[1]: Starting Permit User Sessions...886gitlab # [ 7.881955] systemd[1]: Finished Permit User Sessions.887gitlab # [ 7.923668] systemd[1]: Started Getty on tty1.888gitlab # [ 7.925931] systemd[1]: Reached target Login Prompts.889gitlab # [ 8.094557] nginx-pre-start[839]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok890gitlab # [ 8.096857] nginx-pre-start[839]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful891gitlab # [ 8.105608] systemd[1]: Started Nginx Web Server.892gitlab # [ 8.128111] postgresql-pre-start[841]: The files belonging to this database system will be owned by user "postgres".893gitlab # [ 8.129711] postgresql-pre-start[841]: This user must also own the server process.894gitlab # [ 8.133322] postgresql-pre-start[841]: The database cluster will be initialized with locale "en_US.UTF-8".895gitlab # [ 8.135084] postgresql-pre-start[841]: The default database encoding has accordingly been set to "UTF8".896gitlab # [ 8.136841] postgresql-pre-start[841]: The default text search configuration will be set to "english".897gitlab # [ 8.140367] postgresql-pre-start[841]: Data page checksums are enabled.898gitlab # [ 8.141339] postgresql-pre-start[841]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok899gitlab # [ 8.142611] postgresql-pre-start[841]: creating subdirectories ... ok900gitlab # [ 8.143460] postgresql-pre-start[841]: selecting dynamic shared memory implementation ... posix901gitlab # [ 8.147116] redis-server[836]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.902gitlab # [ 8.149799] redis-server[836]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo903gitlab # [ 8.151240] redis-server[836]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=836, just started904gitlab # [ 8.153980] redis-server[836]: Configuration loaded905gitlab # [ 8.155025] redis[836]: monotonic clock: ARM CNTVCT @ 1000 ticks/us906gitlab # [ 8.157814] redis[836]: Running mode=standalone, port=0.907gitlab # [ 8.158994] redis[836]: Server initialized908gitlab # [ 8.160949] redis[836]: Ready to accept connections unix909gitlab # [ 8.161958] systemd[1]: Started Redis Server - redis-gitlab.910gitlab # [ 8.162813] redis[836]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.911gitlab # [ 8.201325] postgresql-pre-start[841]: selecting default "max_connections" ... 100912gitlab # [ 8.218293] cfg80211: Loading compiled-in X.509 certificates for regulatory database913gitlab # [ 8.233922] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'914gitlab # [ 8.234458] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'915gitlab # [ 8.238212] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2916gitlab # [ 8.238548] cfg80211: failed to load regulatory.db917gitlab # [ 8.241389] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3918gitlab # [ 8.291782] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.919gitlab # [ 8.293197] systemd[1]: Stopped Virtual Console Setup.920gitlab # [ 8.293948] systemd[1]: systemd-vconsole-setup.service: Consumed 45ms CPU time over 1.901s wall clock time, 2.9M memory peak, 1.1M read from disk.921gitlab # [ 8.295624] systemd[1]: Stopping Virtual Console Setup...922gitlab # [ 8.307982] systemd[1]: Starting Virtual Console Setup...923gitlab # [ 8.309988] postgresql-pre-start[841]: selecting default "shared_buffers" ... 128MB924gitlab # [ 8.314313] 8021q: adding VLAN 0 to HW filter on device eth0925gitlab # [ 8.314825] dhcpcd[812]: libudev: received NULL device926gitlab # [ 8.315808] dhcpcd[812]: eth0: waiting for carrier927gitlab # [ 8.316695] dhcpcd[812]: libudev: received NULL device928gitlab # [ 8.317834] dhcpcd[812]: eth0: carrier acquired929gitlab # [ 8.318663] dhcpcd[812]: DUID 00:01:00:01:32:2c:04:b7:52:54:00:12:34:56930gitlab # [ 8.320023] dhcpcd[812]: eth0: IAID 00:12:34:56931gitlab # [ 8.320901] dhcpcd[812]: eth0: adding address fe80::5054:ff:fe12:3456932gitlab # [ 8.327537] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.933gitlab # [ 8.330166] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.934gitlab # [ 8.331578] systemd[1]: Stopped Virtual Console Setup.935gitlab # [ 8.340122] systemd[1]: Starting Virtual Console Setup...936gitlab # [ 8.378745] systemd-logind[659]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)937gitlab # [ 8.380966] systemd-vconsole-setup[878]: Configuration of first virtual console was skipped, ignoring remaining ones.938gitlab # [ 8.383482] systemd[1]: Finished Virtual Console Setup.939gitlab # [ 8.493572] dhcpcd[812]: eth0: soliciting a DHCP lease940gitlab # [ 8.516751] dhcpcd[812]: eth0: offered 10.0.2.15 from 10.0.2.2941gitlab # [ 8.540603] dhcpcd[812]: eth0: probing address 10.0.2.15/24942gitlab # [ 8.695860] systemd[1]: Started gitlab-workhorse.service.943gitlab # [ 8.840209] postgresql-pre-start[841]: selecting default time zone ... UTC944gitlab # [ 8.844406] postgresql-pre-start[841]: creating configuration files ... ok945gitlab # [ 8.989762] gitlab-workhorse[893]: time="2026-09-03T10:13:11Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4946gitlab # [ 8.996404] gitlab-workhorse[893]: time="2026-09-03T10:13:11Z" level=info msg="Using redis/go-redis"947gitlab # [ 8.997918] gitlab-workhorse[893]: time="2026-09-03T10:13:11Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix948gitlab # [ 9.145891] postgresql-pre-start[841]: running bootstrap script ... ok949gitlab # [ 9.880593] postgresql-pre-start[841]: performing post-bootstrap initialization ... ok950gitlab # [ 10.238664] postgresql-pre-start[841]: syncing data to disk ... ok951gitlab # [ 10.241553] postgresql-pre-start[841]: initdb: warning: enabling "trust" authentication for local connections952gitlab # [ 10.246804] postgresql-pre-start[841]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.953gitlab # [ 10.253232] postgresql-pre-start[841]: Success. You can now start the database server using:954gitlab # [ 10.256726] postgresql-pre-start[841]: pg_ctl -D /var/lib/postgresql/18 -l logfile start955gitlab # [ 10.418345] postgres[919]: [919] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit956gitlab # [ 10.424854] postgres[919]: [919] LOG: listening on IPv6 address "::1", port 5432957gitlab # [ 10.429359] postgres[919]: [919] LOG: listening on IPv4 address "127.0.0.1", port 5432958gitlab # [ 10.433374] postgres[919]: [919] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"959gitlab # [ 10.439098] postgres[933]: [933] LOG: database system was shut down at 2026-09-03 10:13:12 GMT960gitlab # [ 10.443726] postgres[919]: [919] LOG: database system is ready to accept connections961gitlab # [ 10.449014] systemd[1]: Started PostgreSQL Server.962gitlab # [ 10.451273] systemd[1]: Starting PostgreSQL Setup Scripts...963gitlab # [ 10.480171] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[926]: mkdir -p /var/gitlab/state/home/.ssh: OK964gitlab # [ 10.485119] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[926]: chmod 700 /var/gitlab/state/home/.ssh: OK965gitlab # [ 10.588679] systemd[1]: Finished gitlab-config.service.966gitlab # [ 10.592796] systemd[1]: Started gitaly.service.967gitlab # [ 10.783966] postgresql-setup-start[953]: CREATE DATABASE968gitlab # [ 10.833321] postgresql-setup-start[972]: CREATE ROLE969gitlab # [ 10.857335] postgresql-setup-start[974]: ALTER DATABASE970gitlab # [ 10.917519] postgresql-setup-start[980]: CREATE ROLE971gitlab # [ 10.922661] systemd[1]: Finished PostgreSQL Setup Scripts.972gitlab # [ 10.925210] systemd[1]: Reached target PostgreSQL.973gitlab # [ 10.929881] systemd[1]: Starting gitlab-db-config.service...974gitlab # [ 10.935195] systemd[1]: Starting gitlab-postgresql.service...975gitlab # [ 11.143528] gitlab-postgresql-start[994]: CREATE DATABASE976gitlab # [ 11.197968] gitlab-postgresql-start[1000]: CREATE EXTENSION977gitlab # [ 11.204721] dhcpcd[812]: eth0: soliciting an IPv6 router978gitlab # [ 11.205946] dhcpcd[812]: eth0: Router Advertisement from fe80::2979gitlab # [ 11.207120] dhcpcd[812]: eth0: adding address fec0::5054:ff:fe12:3456/64980gitlab # [ 11.208803] dhcpcd[812]: eth0: adding route to fec0::/64981gitlab # [ 11.209883] dhcpcd[812]: eth0: adding default route via fe80::2982gitlab # [ 11.294368] gitlab-postgresql-start[1002]: CREATE EXTENSION983gitlab # [ 11.299147] systemd[1]: Finished gitlab-postgresql.service.984gitlab # [ 11.462393] gitaly[958]: time="2026-09-03T10:13:14.308Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=958985gitlab # [ 11.467535] gitaly[958]: time="2026-09-03T10:13:14.308Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=958986gitlab # [ 11.472172] gitaly[958]: time="2026-09-03T10:13:14.308Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=958987gitlab # [ 11.474298] gitaly[958]: time="2026-09-03T10:13:14.308Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=958988gitlab # [ 11.475985] gitaly[958]: time="2026-09-03T10:13:14.308Z" level=info msg="Starting Gitaly" pid=958 version=989gitlab # [ 11.685977] gitaly[958]: time="2026-09-03T10:13:14.535Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=226 pid=958990gitlab # [ 11.688075] gitaly[958]: time="2026-09-03T10:13:14.536Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=958991gitlab # [ 11.689688] gitaly[958]: time="2026-09-03T10:13:14.536Z" level=info msg="finished initializing command factory" duration_ms=0 pid=958992gitlab # [ 11.691208] gitaly[958]: time="2026-09-03T10:13:14.536Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-958/git-exec-2734928265.d/git pid=958993gitlab # [ 11.693790] gitaly[958]: time="2026-09-03T10:13:14.543Z" level=info msg="finished detecting git version" duration_ms=7 pid=958994gitlab # [ 11.695250] gitaly[958]: time="2026-09-03T10:13:14.543Z" level=info msg="using Git version" pid=958 version=2.53.g995gitlab # [ 11.696647] gitaly[958]: time="2026-09-03T10:13:14.544Z" level=info msg="clearing disk cache object folder" pid=958 storage=default996gitlab # [ 11.698105] gitaly[958]: time="2026-09-03T10:13:14.544Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3377655133 pid=958 storage=default997gitlab # [ 11.700384] gitaly[958]: time="2026-09-03T10:13:14.544Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3377655133 pid=958 storage=default998gitlab # [ 11.702857] gitaly[958]: time="2026-09-03T10:13:14.544Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=958999gitlab # [ 11.705213] gitaly[958]: time="2026-09-03T10:13:14.544Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9581000gitlab # [ 11.707637] gitaly[958]: time="2026-09-03T10:13:14.544Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3377655133 clear_duration_ms=0 pid=958 storage=default1001gitlab # [ 11.730712] gitaly[958]: time="2026-09-03T10:13:14.578Z" level=info msg="finished tempdir cleaner walk" pid=958 storage=default time_ms=01002gitlab # [ 11.732502] gitaly[958]: time="2026-09-03T10:13:14.578Z" level=info msg="counting repositories" pid=9581003gitlab # [ 11.733722] gitaly[958]: time="2026-09-03T10:13:14.578Z" level=info msg="starting to count repositories" pid=958 storage_path=/var/gitlab/state/repositories1004gitlab # [ 11.735484] gitaly[958]: time="2026-09-03T10:13:14.578Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=958 storage_path=/var/gitlab/state/repositories1005gitlab # [ 11.737658] gitaly[958]: time="2026-09-03T10:13:14.578Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9581006gitlab # [ 11.739207] gitaly[958]: time="2026-09-03T10:13:14.578Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9581007gitlab # [ 11.741344] gitaly[958]: time="2026-09-03T10:13:14.578Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=958 schema=unix1008gitlab # [ 11.743323] gitaly[958]: time="2026-09-03T10:13:14.579Z" level=info msg="listening at address" address=/run/gitaly/gitaly-958/sock.d/intern pid=958 schema=unix1009gitlab # [ 11.745864] gitaly[958]: time="2026-09-03T10:13:14.580Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9581010gitlab # [ 11.747428] gitaly[958]: time="2026-09-03T10:13:14.580Z" level=info msg="Started Gitaly" duration_ms=272 pid=9581011gitlab # [ 11.752263] gitaly[958]: time="2026-09-03T10:13:14.580Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9581012gitlab # [ 11.753917] gitaly[958]: time="2026-09-03T10:13:14.581Z" level=info msg="maintenance: daily scheduled" pid=958 scheduled="2026-09-03 12:00:00 +0000 UTC"1013gitlab # [ 12.270695] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[992]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1014gitlab # [ 12.280817] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[992]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1015gitlab # [ 12.296364] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[992]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1016gitlab # [ 13.461175] dhcpcd[812]: eth0: leased 10.0.2.15 for 86400 seconds1017gitlab # [ 13.463937] dhcpcd[812]: eth0: adding route to 10.0.2.0/241018gitlab # [ 13.466358] dhcpcd[812]: eth0: adding default route via 10.0.2.21019gitlab # [ 13.663877] systemd[1]: Started DHCP Client.1020gitlab # [ 13.666209] systemd[1]: Reached target Network is Online.1021gitlab # [ 15.208357] gitaly[958]: time="2026-09-03T10:13:18.057Z" level=info msg="License database preloaded" duration_ms=3748 pid=9581022gitlab # [ 58.090664] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[992]: W, [2026-09-03T10:14:00.939268 #992] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1023gitlab # [ 59.088257] gitlab-workhorse[893]: time="2026-09-03T10:14:01Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1024gitlab # [ 61.200792] postgres[1510]: [1510] ERROR: relation "feature_gates" does not exist at character 5231025gitlab # [ 61.201266] postgres[1510]: [1510] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1026gitlab # [ 61.201393] postgres[1510]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1027gitlab # [ 61.201494] postgres[1510]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1028gitlab # [ 61.201605] postgres[1510]: attidentity AS identity,1029gitlab # [ 61.201717] postgres[1510]: attgenerated as attgenerated1030gitlab # [ 61.201830] postgres[1510]: FROM pg_attribute a1031gitlab # [ 61.201927] postgres[1510]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1032gitlab # [ 61.202045] postgres[1510]: LEFT JOIN pg_type t ON a.atttypid = t.oid1033gitlab # [ 61.202161] postgres[1510]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1034gitlab # [ 61.202298] postgres[1510]: WHERE a.attrelid = '"feature_gates"'::regclass1035gitlab # [ 61.202395] postgres[1510]: AND a.attnum > 0 AND NOT a.attisdropped1036gitlab # [ 61.202509] postgres[1510]: ORDER BY a.attnum1037gitlab # [ 61.202620] postgres[1510]:1038gitlab # [ 66.459085] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1561]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1039gitlab # [ 66.459528] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1561]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1040gitlab # [ 69.100154] gitlab-workhorse[893]: time="2026-09-03T10:14:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1041gitlab # [ 79.112991] gitlab-workhorse[893]: time="2026-09-03T10:14:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1042gitlab # [ 89.113150] gitlab-workhorse[893]: time="2026-09-03T10:14:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1043gitlab # [ 95.174760] postgres[1664]: [1664] ERROR: relation "table_without_models" does not exist1044gitlab # [ 95.175149] postgres[1664]: [1664] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1045gitlab # [ 99.123053] gitlab-workhorse[893]: time="2026-09-03T10:14:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1046gitlab # [ 109.123502] gitlab-workhorse[893]: time="2026-09-03T10:14:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1047gitlab # [ 119.132781] gitlab-workhorse[893]: time="2026-09-03T10:15:01Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1048gitlab # [ 129.141394] gitlab-workhorse[893]: time="2026-09-03T10:15:11Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1049gitlab # [ 131.671918] systemd[1]: Finished gitlab-db-config.service.1050gitlab # [ 131.679679] systemd[1]: Started gitlab-sidekiq.service.1051gitlab # [ 131.688695] systemd[1]: Starting gitlab.service...1052gitlab # [ 132.278461] bundle[2103]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1053gitlab # [ 132.284983] bundle[2103]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1054gitlab # [ 132.290812] bundle[2103]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1055gitlab # [ 136.878944] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.699Z","pid":2103,"message":"* Enabling systemd notification integration"}1056gitlab # [ 136.879411] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.727Z","pid":2103,"message":"Puma starting in cluster mode..."}1057gitlab # [ 136.879547] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1058gitlab # [ 136.879684] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1059gitlab # [ 136.879799] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Min threads: 0"}1060gitlab # [ 136.879895] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Max threads: 4"}1061gitlab # [ 136.900091] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Environment: production"}1062gitlab # [ 136.906670] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Master PID: 2103"}1063gitlab # [ 136.906759] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Workers: 2"}1064gitlab # [ 136.906799] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1065gitlab # [ 136.906839] bundle[2103]: {"timestamp":"2026-09-03T10:15:19.728Z","pid":2103,"message":"* Preloading application"}1066gitlab # [ 136.907266] sidekiq-cluster[2102]: {"severity":"INFO","time":"2026-09-03T10:15:19.716Z","message":"Starting cluster with 1 processes"}1067gitlab # [ 139.142110] gitlab-workhorse[893]: time="2026-09-03T10:15:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1068gitlab # [ 149.158574] gitlab-workhorse[893]: time="2026-09-03T10:15:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1069gitlab # [ 159.166252] gitlab-workhorse[893]: time="2026-09-03T10:15:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1070gitlab # [ 169.187164] gitlab-workhorse[893]: time="2026-09-03T10:15:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1071gitlab # [ 176.563523] bundle[2103]: W, [2026-09-03T10:15:59.412294 #2103] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1072gitlab # [ 179.196441] gitlab-workhorse[893]: time="2026-09-03T10:16:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1073gitlab # [ 181.930377] sidekiq-cluster[2153]: 2026-09-03T10:16:04.778Z pid=2153 tid=xq9 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2153, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432963584}1074gitlab # [ 182.225693] sidekiq-cluster[2153]: W, [2026-09-03T10:16:05.075493 #2153] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1075gitlab # [ 182.294105] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:05.143Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1076gitlab # [ 186.191106] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.039Z","message":"GitLab reliable fetch activated!"}1077gitlab # [ 186.196270] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.040Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1078gitlab # [ 186.263349] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.113Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1079gitlab # [ 186.270667] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.118Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1080gitlab # [ 186.277899] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.123Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1081gitlab # [ 186.284189] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.126Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1082gitlab # [ 186.290189] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.131Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1083gitlab # [ 186.295098] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.134Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1084gitlab # [ 186.299759] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.137Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1085gitlab # [ 186.303730] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.140Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1086gitlab # [ 186.307524] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.142Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1087gitlab # [ 186.311158] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.145Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1088gitlab # [ 186.315043] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.147Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1089gitlab # [ 186.318602] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.150Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1090gitlab # [ 186.321727] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.153Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1091gitlab # [ 186.324916] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.155Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1092gitlab # [ 186.327934] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.157Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1093gitlab # [ 186.330969] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.160Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1094gitlab # [ 186.333713] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.164Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1095gitlab # [ 186.336265] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.168Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1096gitlab # [ 186.338957] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.171Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1097gitlab # [ 186.341604] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.174Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1098gitlab # [ 186.344392] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.177Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1099gitlab # [ 186.347149] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.179Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1100gitlab # [ 186.349627] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.182Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1101gitlab # [ 186.352000] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.184Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1102gitlab # [ 186.354323] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.187Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1103gitlab # [ 186.356617] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.191Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1104gitlab # [ 186.358788] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.194Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1105gitlab # [ 186.361091] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.197Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1106gitlab # [ 186.363386] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.201Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1107gitlab # [ 186.365824] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.204Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1108gitlab # [ 186.367925] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.207Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1109gitlab # [ 186.370370] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.210Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1110gitlab # [ 186.372499] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.212Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1111gitlab # [ 186.374689] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.215Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1112gitlab # [ 186.376899] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.217Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1113gitlab # [ 186.379014] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.219Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1114gitlab # [ 186.381258] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.221Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1115gitlab # [ 186.383330] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.224Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1116gitlab # [ 186.385520] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.226Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1117gitlab # [ 186.387551] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.228Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1118gitlab # [ 186.389900] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.230Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1119gitlab # [ 186.392040] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.232Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1120gitlab # [ 186.394216] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.234Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1121gitlab # [ 186.396300] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.236Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1122gitlab # [ 186.398499] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.238Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1123gitlab # [ 186.400846] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.240Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1124gitlab # [ 186.403002] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.242Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1125gitlab # [ 186.405233] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.244Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1126gitlab # [ 186.407342] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.247Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1127gitlab # [ 186.409567] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.249Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1128gitlab # [ 186.411684] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.251Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1129gitlab # [ 186.413959] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.254Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1130gitlab # [ 186.416179] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.265Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1131gitlab # [ 186.418925] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.268Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1132gitlab # [ 186.421529] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.271Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1133gitlab # [ 186.424035] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.274Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1134gitlab # [ 186.426300] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.276Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1135gitlab # [ 186.428731] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.278Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1136gitlab # [ 186.431066] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.280Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1137gitlab # [ 186.434556] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.282Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1138gitlab # [ 186.436741] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.284Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1139gitlab # [ 186.438831] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.286Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1140gitlab # [ 186.441193] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.289Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1141gitlab # [ 186.443462] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.291Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1142gitlab # [ 186.445934] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.294Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1143gitlab # [ 186.448260] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.296Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1144gitlab # [ 186.450409] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.298Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1145gitlab # [ 186.452722] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.300Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1146gitlab # [ 186.454965] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.301Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1147gitlab # [ 186.457315] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.303Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1148gitlab # [ 186.459489] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.305Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1149gitlab # [ 186.461754] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.307Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1150gitlab # [ 186.463999] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.308Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1151gitlab # [ 186.466389] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.310Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1152gitlab # [ 186.468787] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.312Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1153gitlab # [ 186.471149] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.314Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1154gitlab # [ 186.473484] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.316Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1155gitlab # [ 186.475953] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.318Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1156gitlab # [ 186.478337] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.320Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1157gitlab # [ 186.480549] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.322Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1158gitlab # [ 186.482906] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.323Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1159gitlab # [ 186.485422] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.325Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1160gitlab # [ 186.487741] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.328Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1161gitlab # [ 186.490147] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.330Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1162gitlab # [ 186.492603] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.332Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1163gitlab # [ 186.494811] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.334Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1164gitlab # [ 186.497259] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.337Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1165gitlab # [ 186.499659] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.339Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1166gitlab # [ 186.502152] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.341Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1167gitlab # [ 186.504523] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.343Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1168gitlab # [ 186.506844] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.344Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1169gitlab # [ 186.509163] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.346Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1170gitlab # [ 186.511426] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.348Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1171gitlab # [ 186.513955] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.350Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1172gitlab # [ 186.515943] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.353Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1173gitlab # [ 186.518399] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.355Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1174gitlab # [ 186.520695] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.358Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1175gitlab # [ 186.522916] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.360Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1176gitlab # [ 186.525205] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.373Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1177gitlab # [ 186.527405] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.375Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1178gitlab # [ 186.529683] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.377Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1179gitlab # [ 186.531925] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.381Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1180gitlab # [ 186.534092] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.384Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1181gitlab # [ 186.536838] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.386Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1182gitlab # [ 186.539531] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.389Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1183gitlab # [ 186.541973] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.391Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1184gitlab # [ 186.594180] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.444Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1185gitlab # [ 186.597903] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.447Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1186gitlab # [ 186.601227] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:09.450Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1187gitlab # [ 189.206307] gitlab-workhorse[893]: time="2026-09-03T10:16:12Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1188gitlab # [ 199.215370] gitlab-workhorse[893]: time="2026-09-03T10:16:22Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1189gitlab # [ 209.225348] gitlab-workhorse[893]: time="2026-09-03T10:16:32Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1190gitlab # [ 219.234262] gitlab-workhorse[893]: time="2026-09-03T10:16:42Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1191gitlab # [ 228.036417] postgres[2917]: [2917] ERROR: relation "table_without_models" does not exist1192gitlab # [ 228.036885] postgres[2917]: [2917] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1193gitlab # [ 229.011904] postgres[2928]: [2928] ERROR: relation "table_without_models" does not exist1194gitlab # [ 229.012334] postgres[2928]: [2928] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1195gitlab # [ 229.242240] gitlab-workhorse[893]: time="2026-09-03T10:16:52Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1196gitlab # [ 233.041387] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:55.887Z","message":"Booted Rails 7.2.3.1 application in production environment"}1197gitlab # [ 233.048552] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:55.898Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1198gitlab # [ 233.054792] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:55.898Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1199gitlab # [ 233.060297] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:55.898Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1200gitlab # [ 233.066128] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:55.900Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1201gitlab # [ 235.765541] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:58.614Z","message":"Cleaning working queues"}1202gitlab # [ 235.770885] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:58.620Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a4e9d33d9d7349e900dec48d","created_at":"2026-09-03T10:15:13.865Z","trace_propagation_headers":{"sentry-trace":"21e246a71354418298375df19f318607-2d5e4cb0dcd143d0","baggage":"sentry-trace_id=21e246a71354418298375df19f318607,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"40b05d8b9dc47b51bc04c30a26a50cd4","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:13.881Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a4e9d33d9d7349e900dec48d: start","job_status":"start","queue_duration_s":104.738115,"scheduling_latency_s":104.738221}1203gitlab # [ 235.990695] bundle[2103]: {"timestamp":"2026-09-03T10:16:58.839Z","pid":2103,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1204gitlab # [ 235.996546] bundle[2103]: {"timestamp":"2026-09-03T10:16:58.839Z","pid":2103,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1205gitlab # [ 236.001629] bundle[2103]: {"timestamp":"2026-09-03T10:16:58.839Z","pid":2103,"message":"! #\u003cThread:0x0000efeb38b1ba48 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1206gitlab # [ 236.017206] bundle[2103]: {"timestamp":"2026-09-03T10:16:58.839Z","pid":2103,"message":"! #\u003cThread:0x0000efeb383eaaa8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1207gitlab # [ 236.031647] bundle[2103]: {"timestamp":"2026-09-03T10:16:58.839Z","pid":2103,"message":"Use Ctrl-C to stop"}1208gitlab # [ 236.034742] bundle[2103]: {"timestamp":"2026-09-03T10:16:58.840Z","pid":2103,"message":"Sending status to systemd every 1.0 sec"}1209gitlab # [ 236.100296] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:16:58.949Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a4e9d33d9d7349e900dec48d","created_at":"2026-09-03T10:15:13.865Z","trace_propagation_headers":{"sentry-trace":"21e246a71354418298375df19f318607-2d5e4cb0dcd143d0","baggage":"sentry-trace_id=21e246a71354418298375df19f318607,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"40b05d8b9dc47b51bc04c30a26a50cd4","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:15:13.881Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a4e9d33d9d7349e900dec48d: done: 0.329386 sec","job_status":"done","queue_duration_s":104.738115,"scheduling_latency_s":104.738221,"redis_calls":28,"redis_duration_s":0.010921,"redis_read_bytes":408,"redis_write_bytes":3436,"redis_feature_flag_calls":17,"redis_feature_flag_duration_s":0.005915,"redis_feature_flag_read_bytes":398,"redis_feature_flag_write_bytes":2189,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003321,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":13,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.175536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1842573,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.329386,"completed_at":"2026-09-03T10:16:58.949Z","load_balancing_strategy":"primary","db_duration_s":0.073769,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1210gitlab # [ 236.689297] bundle[2103]: {"timestamp":"2026-09-03T10:16:59.538Z","pid":2103,"message":"- Worker 0 (PID: 3046) booted in 0.63s, phase: 0"}1211gitlab # [ 236.858697] bundle[2103]: {"timestamp":"2026-09-03T10:16:59.708Z","pid":2103,"message":"- Worker 1 (PID: 3050) booted in 0.77s, phase: 0"}1212gitlab # [ 236.866102] systemd[1]: Started gitlab.service.1213gitlab # [ 236.867136] systemd[1]: Reached target Common target for all GitLab services..1214gitlab # [ 236.873637] systemd[1]: Starting setup-gitlab.service...1215gitlab # [ 236.930215] setup-gitlab-start[3083]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1216gitlab # [ 241.489790] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:17:04 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 45001217gitlab # [ 241.497811] setup-gitlab-start[3083]: + TOKEN=nixbot-test-token-of-20-chars1218gitlab # [ 241.498968] setup-gitlab-start[3083]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1219gitlab # [ 241.525932] runuser[3113]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1220gitlab # [ 242.585466] setup-gitlab-start[3114]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1221gitlab # [ 242.592662] setup-gitlab-start[3114]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1222gitlab # [ 242.599083] setup-gitlab-start[3114]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1223gitlab # [ 243.670156] setup-gitlab-start[3114]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1224gitlab # [ 243.670557] setup-gitlab-start[3114]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1225gitlab # [ 243.671902] setup-gitlab-start[3114]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1226gitlab # [ 247.996763] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:10.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f279375f936112c6129c8a10","created_at":"2026-09-03T10:17:10.747Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"916226fae0d0401f43ab7cade710858f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.840Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f279375f936112c6129c8a10: start","job_status":"start","queue_duration_s":0.003859,"scheduling_latency_s":0.003933}1227gitlab # [ 251.442406] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:14.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f279375f936112c6129c8a10","created_at":"2026-09-03T10:17:10.747Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"916226fae0d0401f43ab7cade710858f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.840Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f279375f936112c6129c8a10: done: 3.445416 sec","job_status":"done","queue_duration_s":0.003859,"scheduling_latency_s":0.003933,"redis_calls":565,"redis_duration_s":2.159716,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007134,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01006,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":2.142522,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.097744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00347564,"duration_s":3.445416,"completed_at":"2026-09-03T10:17:14.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.161487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1228gitlab # [ 251.546112] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:14.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"26a44cfa710b5373ebd26fa6","created_at":"2026-09-03T10:17:10.884Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93d6b4a750b374c8b115603bf0c20b97","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.940Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-26a44cfa710b5373ebd26fa6: start","job_status":"start","queue_duration_s":3.454879,"scheduling_latency_s":3.454932}1229gitlab # [ 251.662462] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:14.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"26a44cfa710b5373ebd26fa6","created_at":"2026-09-03T10:17:10.884Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93d6b4a750b374c8b115603bf0c20b97","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:10.940Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-26a44cfa710b5373ebd26fa6: done: 0.11623 sec","job_status":"done","queue_duration_s":3.454879,"scheduling_latency_s":3.454932,"redis_calls":10,"redis_duration_s":0.019903999999999998,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004611,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003157,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0027452,"duration_s":0.11623,"completed_at":"2026-09-03T10:17:14.511Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1230gitlab # [ 251.726313] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:14.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0e9ad925353911e7629cacf8","created_at":"2026-09-03T10:17:10.957Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b2b9a1227907982e127643818ddc50a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.017Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0e9ad925353911e7629cacf8: start","job_status":"start","queue_duration_s":3.506774,"scheduling_latency_s":3.506826}1231gitlab # [ 252.064324] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:14.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0e9ad925353911e7629cacf8","created_at":"2026-09-03T10:17:10.957Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b2b9a1227907982e127643818ddc50a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.017Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0e9ad925353911e7629cacf8: done: 0.387728 sec","job_status":"done","queue_duration_s":3.506774,"scheduling_latency_s":3.506826,"redis_calls":12,"redis_duration_s":0.031702,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.019783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007674,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0045304,"exclusive_lock_hold_duration_s":0.24488946,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.387728,"completed_at":"2026-09-03T10:17:14.912Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066372,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1232gitlab # [ 252.152935] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:14.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c7f8789af5dba51ba08d1955","created_at":"2026-09-03T10:17:11.047Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69cbfdcb84443760232c6ccdbde64f1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.079Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c7f8789af5dba51ba08d1955: start","job_status":"start","queue_duration_s":3.850888,"scheduling_latency_s":3.850944}1233gitlab # [ 252.183177] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:15.031Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1234gitlab # [ 252.213467] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:15.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c7f8789af5dba51ba08d1955","created_at":"2026-09-03T10:17:11.047Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69cbfdcb84443760232c6ccdbde64f1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.079Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c7f8789af5dba51ba08d1955: done: 0.132132 sec","job_status":"done","queue_duration_s":3.850888,"scheduling_latency_s":3.850944,"redis_calls":10,"redis_duration_s":0.023038,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00503,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015173,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002835,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00242294,"duration_s":0.132132,"completed_at":"2026-09-03T10:17:15.062Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1235gitlab # [ 252.276526] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:15.066Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"04ff767be6b92776f8fee5b8","created_at":"2026-09-03T10:17:11.143Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"671669082eb4b7c064c03f6c8f4cdf10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.177Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-04ff767be6b92776f8fee5b8: start","job_status":"start","queue_duration_s":3.888392,"scheduling_latency_s":3.888442}1236gitlab # [ 253.201710] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"04ff767be6b92776f8fee5b8","created_at":"2026-09-03T10:17:11.143Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"671669082eb4b7c064c03f6c8f4cdf10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.177Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-04ff767be6b92776f8fee5b8: done: 0.982687 sec","job_status":"done","queue_duration_s":3.888392,"scheduling_latency_s":3.888442,"redis_calls":14,"redis_duration_s":0.048941,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.032707,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00952,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.205,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.194,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.50561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00204668,"exclusive_lock_hold_duration_s":0.86311822,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.982687,"completed_at":"2026-09-03T10:17:16.048Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.450773,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1237gitlab # [ 253.296957] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"cae55ab725a571a99c816ff7","created_at":"2026-09-03T10:17:11.204Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be4c6a2c963194085c781f3ec080f793","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:17:11.257Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-cae55ab725a571a99c816ff7: start","job_status":"start","queue_duration_s":4.829755,"scheduling_latency_s":4.829798}1238gitlab # [ 253.331289] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.181Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1239gitlab # [ 253.375306] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.224Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"cae55ab725a571a99c816ff7","created_at":"2026-09-03T10:17:11.204Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be4c6a2c963194085c781f3ec080f793","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:17:11.257Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-cae55ab725a571a99c816ff7: done: 0.137021 sec","job_status":"done","queue_duration_s":4.829755,"scheduling_latency_s":4.829798,"redis_calls":10,"redis_duration_s":0.027019,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00407254,"duration_s":0.137021,"completed_at":"2026-09-03T10:17:16.224Z","load_balancing_strategy":"primary","db_duration_s":0.003276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1240gitlab # [ 253.444400] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.237Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0297dfa87da6e6fca372aeae","created_at":"2026-09-03T10:17:11.294Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7969df5431a075e0bc64feb79772e56b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:17:11.350Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0297dfa87da6e6fca372aeae: start","job_status":"start","queue_duration_s":4.886707,"scheduling_latency_s":4.886763}1241gitlab # [ 253.461728] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.300Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1242gitlab # [ 253.491997] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0297dfa87da6e6fca372aeae","created_at":"2026-09-03T10:17:11.294Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7969df5431a075e0bc64feb79772e56b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:17:11.350Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0297dfa87da6e6fca372aeae: done: 0.104252 sec","job_status":"done","queue_duration_s":4.886707,"scheduling_latency_s":4.886763,"redis_calls":10,"redis_duration_s":0.031485,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004569,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.008164,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0048783,"duration_s":0.104252,"completed_at":"2026-09-03T10:17:16.341Z","load_balancing_strategy":"primary","db_duration_s":0.004842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 253.548696] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.360Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"63b0237d93d23a08c73ced67","created_at":"2026-09-03T10:17:11.390Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.449Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-63b0237d93d23a08c73ced67: start","job_status":"start","queue_duration_s":4.910197,"scheduling_latency_s":4.910256}1244gitlab # [ 253.688289] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"63b0237d93d23a08c73ced67","created_at":"2026-09-03T10:17:11.390Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.449Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-63b0237d93d23a08c73ced67: done: 0.176657 sec","job_status":"done","queue_duration_s":4.910197,"scheduling_latency_s":4.910256,"redis_calls":19,"redis_duration_s":0.04963,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.01257,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.003537,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.029442,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004081,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00375554,"duration_s":0.176657,"completed_at":"2026-09-03T10:17:16.536Z","load_balancing_strategy":"primary","db_duration_s":0.013305,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1245gitlab # [ 253.763211] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a3b8789582618f6b14929e84","created_at":"2026-09-03T10:17:11.501Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da4b14dc4f204cc72ce3723f19696d16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.555Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a3b8789582618f6b14929e84: start","job_status":"start","queue_duration_s":5.006805,"scheduling_latency_s":5.006862}1246gitlab # [ 253.817123] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.666Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1247gitlab # [ 253.869024] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a3b8789582618f6b14929e84","created_at":"2026-09-03T10:17:11.501Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da4b14dc4f204cc72ce3723f19696d16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.555Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a3b8789582618f6b14929e84: done: 0.155192 sec","job_status":"done","queue_duration_s":5.006805,"scheduling_latency_s":5.006862,"redis_calls":10,"redis_duration_s":0.026869999999999998,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004638,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00387108,"duration_s":0.155192,"completed_at":"2026-09-03T10:17:16.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1248gitlab # [ 253.950641] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.733Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c3172d188f8ad6b9d416a3ee","created_at":"2026-09-03T10:17:11.600Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd9d5c7c8005fa374b89a5dbac0a3918","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.652Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c3172d188f8ad6b9d416a3ee: start","job_status":"start","queue_duration_s":5.080309,"scheduling_latency_s":5.080365}1249gitlab # [ 254.066984] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.915Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c3172d188f8ad6b9d416a3ee","created_at":"2026-09-03T10:17:11.600Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd9d5c7c8005fa374b89a5dbac0a3918","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.652Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c3172d188f8ad6b9d416a3ee: done: 0.182844 sec","job_status":"done","queue_duration_s":5.080309,"scheduling_latency_s":5.080365,"redis_calls":16,"redis_duration_s":0.050531,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.026892,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.020354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00447784,"duration_s":0.182844,"completed_at":"2026-09-03T10:17:16.915Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1250gitlab # [ 254.158609] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:16.920Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"540a2d5791b3e80162e7e590","created_at":"2026-09-03T10:17:11.735Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.783Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-540a2d5791b3e80162e7e590: start","job_status":"start","queue_duration_s":5.136978,"scheduling_latency_s":5.137045}1251gitlab # [ 254.745333] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"540a2d5791b3e80162e7e590","created_at":"2026-09-03T10:17:11.735Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.783Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-540a2d5791b3e80162e7e590: done: 0.671925 sec","job_status":"done","queue_duration_s":5.136978,"scheduling_latency_s":5.137045,"redis_calls":71,"redis_duration_s":0.19515,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.029243,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00408,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.153907,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00792,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.296507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00454542,"duration_s":0.671925,"completed_at":"2026-09-03T10:17:17.592Z","load_balancing_strategy":"primary","db_duration_s":0.027097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1252gitlab # [ 254.759001] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2bc5313a412f9c40bec28d06","created_at":"2026-09-03T10:17:11.817Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0961a809fd9b16cda437146353daf7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.913Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2bc5313a412f9c40bec28d06: start","job_status":"start","queue_duration_s":5.694308,"scheduling_latency_s":5.694368}1253gitlab # [ 254.902708] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.752Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1254gitlab # [ 254.945893] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2bc5313a412f9c40bec28d06","created_at":"2026-09-03T10:17:11.817Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0961a809fd9b16cda437146353daf7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:11.913Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2bc5313a412f9c40bec28d06: done: 0.183581 sec","job_status":"done","queue_duration_s":5.694308,"scheduling_latency_s":5.694368,"redis_calls":10,"redis_duration_s":0.004898,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000889,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00251,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001499,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0008004,"duration_s":0.183581,"completed_at":"2026-09-03T10:17:17.791Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1255gitlab # [ 255.009435] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ae2cca165f613bce4a8d2054","created_at":"2026-09-03T10:17:11.961Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3bac5f69b23deaa63e853b4ce25b543","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.021Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ae2cca165f613bce4a8d2054: start","job_status":"start","queue_duration_s":5.776945,"scheduling_latency_s":5.776998}1256gitlab # [ 255.024120] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ae2cca165f613bce4a8d2054","created_at":"2026-09-03T10:17:11.961Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3bac5f69b23deaa63e853b4ce25b543","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.021Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ae2cca165f613bce4a8d2054: done: 0.064252 sec","job_status":"done","queue_duration_s":5.776945,"scheduling_latency_s":5.776998,"redis_calls":12,"redis_duration_s":0.003715,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000617,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001447,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0015096,"exclusive_lock_hold_duration_s":0.01462352,"duration_s":0.064252,"completed_at":"2026-09-03T10:17:17.862Z","load_balancing_strategy":"primary","db_duration_s":0.00821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1257gitlab # [ 255.066819] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0e7fc1cd52cad3662619590d","created_at":"2026-09-03T10:17:12.065Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02bcddbc043b61dc6ce33a4882916097","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.123Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0e7fc1cd52cad3662619590d: start","job_status":"start","queue_duration_s":5.741337,"scheduling_latency_s":5.741364}1258gitlab # [ 255.097343] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0e7fc1cd52cad3662619590d","created_at":"2026-09-03T10:17:12.065Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02bcddbc043b61dc6ce33a4882916097","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.123Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0e7fc1cd52cad3662619590d: done: 0.08113 sec","job_status":"done","queue_duration_s":5.741337,"scheduling_latency_s":5.741364,"redis_calls":12,"redis_duration_s":0.003387,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001277,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00074434,"exclusive_lock_hold_duration_s":0.04450996,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.08113,"completed_at":"2026-09-03T10:17:17.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1259gitlab # [ 255.147424] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"d3270699ed49b77b1de7e39f","created_at":"2026-09-03T10:17:12.180Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9346642d4d8343a5bc0be40b9731abc2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.248Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-d3270699ed49b77b1de7e39f: start","job_status":"start","queue_duration_s":5.701081,"scheduling_latency_s":5.701112}1260gitlab # [ 255.162076] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"d3270699ed49b77b1de7e39f","created_at":"2026-09-03T10:17:12.180Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9346642d4d8343a5bc0be40b9731abc2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.248Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-d3270699ed49b77b1de7e39f: done: 0.045255 sec","job_status":"done","queue_duration_s":5.701081,"scheduling_latency_s":5.701112,"redis_calls":13,"redis_duration_s":0.002493,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01383956,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.045255,"completed_at":"2026-09-03T10:17:17.994Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003076,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1261gitlab # [ 255.206598] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:17.996Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"51f0e7c3b036fcf9074a0743","created_at":"2026-09-03T10:17:12.287Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb8798f1c8e7df8f8580f38a4769f21d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.340Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-51f0e7c3b036fcf9074a0743: start","job_status":"start","queue_duration_s":5.655488,"scheduling_latency_s":5.655506}1262gitlab # [ 255.222835] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"51f0e7c3b036fcf9074a0743","created_at":"2026-09-03T10:17:12.287Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb8798f1c8e7df8f8580f38a4769f21d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.340Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-51f0e7c3b036fcf9074a0743: done: 0.075844 sec","job_status":"done","queue_duration_s":5.655488,"scheduling_latency_s":5.655506,"redis_calls":12,"redis_duration_s":0.002517,"redis_read_bytes":11,"redis_write_bytes":1354,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":297,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":519,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049756,"exclusive_lock_hold_duration_s":0.04834298,"duration_s":0.075844,"completed_at":"2026-09-03T10:17:18.071Z","load_balancing_strategy":"primary","db_duration_s":0.031116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1263gitlab # [ 255.265180] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"611c3ed19c3a43b80d20b0c3","created_at":"2026-09-03T10:17:12.376Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b255de3934097d9b7a99ec932741c6cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.428Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-611c3ed19c3a43b80d20b0c3: start","job_status":"start","queue_duration_s":5.64581,"scheduling_latency_s":5.645841}1264gitlab # [ 255.279608] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.093Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1265gitlab # [ 255.283424] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"611c3ed19c3a43b80d20b0c3","created_at":"2026-09-03T10:17:12.376Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b255de3934097d9b7a99ec932741c6cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.428Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-611c3ed19c3a43b80d20b0c3: done: 0.029884 sec","job_status":"done","queue_duration_s":5.64581,"scheduling_latency_s":5.645841,"redis_calls":10,"redis_duration_s":0.002051,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033782,"duration_s":0.029884,"completed_at":"2026-09-03T10:17:18.104Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000624,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1266gitlab # [ 255.326196] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.105Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5b52ed198333f28cccef22d9","created_at":"2026-09-03T10:17:12.480Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a857961114e37d6fc14f95c3d23b4c0b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.532Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5b52ed198333f28cccef22d9: start","job_status":"start","queue_duration_s":5.573096,"scheduling_latency_s":5.573116}1267gitlab # [ 255.340812] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5b52ed198333f28cccef22d9","created_at":"2026-09-03T10:17:12.480Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a857961114e37d6fc14f95c3d23b4c0b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.532Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5b52ed198333f28cccef22d9: done: 0.027795 sec","job_status":"done","queue_duration_s":5.573096,"scheduling_latency_s":5.573116,"redis_calls":12,"redis_duration_s":0.002268,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000542,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056636,"exclusive_lock_hold_duration_s":0.0036898,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.027795,"completed_at":"2026-09-03T10:17:18.133Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1268gitlab # [ 255.385031] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"40cf3ac16783700b6ef6f863","created_at":"2026-09-03T10:17:12.570Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f81ca2357b42ee5624029e855f3248d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:17:12.622Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-40cf3ac16783700b6ef6f863: start","job_status":"start","queue_duration_s":5.512697,"scheduling_latency_s":5.512716}1269gitlab # [ 255.398364] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"40cf3ac16783700b6ef6f863","created_at":"2026-09-03T10:17:12.570Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f81ca2357b42ee5624029e855f3248d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:17:12.622Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-40cf3ac16783700b6ef6f863: done: 0.052366 sec","job_status":"done","queue_duration_s":5.512697,"scheduling_latency_s":5.512716,"redis_calls":10,"redis_duration_s":0.002633,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001607,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007218,"duration_s":0.052366,"completed_at":"2026-09-03T10:17:18.187Z","load_balancing_strategy":"primary","db_duration_s":0.004184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 255.439886] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"04ff0adaee105333bdb31b8b","created_at":"2026-09-03T10:17:12.660Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ed0818aaab8364f0f20d25ab632e303","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.723Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-04ff0adaee105333bdb31b8b: start","job_status":"start","queue_duration_s":5.467601,"scheduling_latency_s":5.467644}1271gitlab # [ 255.455048] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.250Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"04ff0adaee105333bdb31b8b","created_at":"2026-09-03T10:17:12.660Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ed0818aaab8364f0f20d25ab632e303","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.723Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-04ff0adaee105333bdb31b8b: done: 0.058515 sec","job_status":"done","queue_duration_s":5.467601,"scheduling_latency_s":5.467644,"redis_calls":12,"redis_duration_s":0.005456,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002008,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000576,"duration_s":0.058515,"completed_at":"2026-09-03T10:17:18.249Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002127,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1272gitlab # [ 255.499504] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ffa5b6cd5a75a3679b2a8229","created_at":"2026-09-03T10:17:12.771Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ae54a3dcdb76dcfcdf0cf9104aafa85","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.823Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ffa5b6cd5a75a3679b2a8229: start","job_status":"start","queue_duration_s":5.429836,"scheduling_latency_s":5.429871}1273gitlab # [ 255.513772] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ffa5b6cd5a75a3679b2a8229","created_at":"2026-09-03T10:17:12.771Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ae54a3dcdb76dcfcdf0cf9104aafa85","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.823Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ffa5b6cd5a75a3679b2a8229: done: 0.073446 sec","job_status":"done","queue_duration_s":5.429836,"scheduling_latency_s":5.429871,"redis_calls":10,"redis_duration_s":0.003483,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000754,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002053,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005911,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.073446,"completed_at":"2026-09-03T10:17:18.326Z","load_balancing_strategy":"primary","db_duration_s":0.026679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1274gitlab # [ 255.557271] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.329Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5b7f1a9fb01a7f5c2d7e728e","created_at":"2026-09-03T10:17:12.858Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3055489b25c1b110af2f3f12431a1632","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.896Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5b7f1a9fb01a7f5c2d7e728e: start","job_status":"start","queue_duration_s":5.432779,"scheduling_latency_s":5.432821}1275gitlab # [ 255.572221] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5b7f1a9fb01a7f5c2d7e728e","created_at":"2026-09-03T10:17:12.858Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3055489b25c1b110af2f3f12431a1632","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:17:12.896Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5b7f1a9fb01a7f5c2d7e728e: done: 0.036187 sec","job_status":"done","queue_duration_s":5.432779,"scheduling_latency_s":5.432821,"redis_calls":10,"redis_duration_s":0.002208,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000501,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038592,"duration_s":0.036187,"completed_at":"2026-09-03T10:17:18.365Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 255.615708] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.367Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2ac18568b318f0272ca19f29","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2ac18568b318f0272ca19f29: start","job_status":"start","queue_duration_s":1.876178,"scheduling_latency_s":1.876208}1277gitlab # [ 255.695213] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.545Z","message":"No existing merge request to be cleaned up."}1278gitlab # [ 255.712339] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.561Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2ac18568b318f0272ca19f29","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2ac18568b318f0272ca19f29: done: 0.194247 sec","job_status":"done","queue_duration_s":1.876178,"scheduling_latency_s":1.876208,"redis_calls":13,"redis_duration_s":0.00345,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001252,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.177866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047154,"duration_s":0.194247,"completed_at":"2026-09-03T10:17:18.561Z","load_balancing_strategy":"primary","db_duration_s":0.00655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1279gitlab # [ 255.784141] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.563Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0534e9ec3f85b31f2fc7d38","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0534e9ec3f85b31f2fc7d38: start","job_status":"start","queue_duration_s":2.072286,"scheduling_latency_s":2.072315}1280gitlab # [ 255.798546] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.576Z","message":"No existing merge request to be cleaned up."}1281gitlab # [ 255.800419] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.587Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0534e9ec3f85b31f2fc7d38","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0534e9ec3f85b31f2fc7d38: done: 0.023824 sec","job_status":"done","queue_duration_s":2.072286,"scheduling_latency_s":2.072315,"redis_calls":10,"redis_duration_s":0.003208,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023824,"completed_at":"2026-09-03T10:17:18.587Z","load_balancing_strategy":"primary","db_duration_s":0.002708,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 255.840207] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.589Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4517e30829afb1958fbabd1e","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4517e30829afb1958fbabd1e: start","job_status":"start","queue_duration_s":2.097748,"scheduling_latency_s":2.097772}1283gitlab # [ 255.852622] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.602Z","message":"No existing merge request to be cleaned up."}1284gitlab # [ 255.854323] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.612Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4517e30829afb1958fbabd1e","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4517e30829afb1958fbabd1e: done: 0.023141 sec","job_status":"done","queue_duration_s":2.097748,"scheduling_latency_s":2.097772,"redis_calls":10,"redis_duration_s":0.0029519999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001778,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023141,"completed_at":"2026-09-03T10:17:18.612Z","load_balancing_strategy":"primary","db_duration_s":0.003474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1285gitlab # [ 255.894741] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.614Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"973251c417c7dffa4d04e153","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-973251c417c7dffa4d04e153: start","job_status":"start","queue_duration_s":2.122515,"scheduling_latency_s":2.122539}1286gitlab # [ 255.907123] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.657Z","message":"No existing merge request to be cleaned up."}1287gitlab # [ 255.908857] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.668Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"973251c417c7dffa4d04e153","created_at":"2026-09-03T10:17:16.434Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-20b2e30d179c4a41","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"adb2db790dc904e4b0e5f07284100b5f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:16.491Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-973251c417c7dffa4d04e153: done: 0.054979 sec","job_status":"done","queue_duration_s":2.122515,"scheduling_latency_s":2.122539,"redis_calls":10,"redis_duration_s":0.005488,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00209,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003398,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054979,"completed_at":"2026-09-03T10:17:18.668Z","load_balancing_strategy":"primary","db_duration_s":0.040896,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 255.947632] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624b0cfdd99a7719ae4c018f","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624b0cfdd99a7719ae4c018f: start","job_status":"start","queue_duration_s":1.12997,"scheduling_latency_s":1.129999}1289gitlab # [ 255.960343] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624b0cfdd99a7719ae4c018f","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624b0cfdd99a7719ae4c018f: done: 0.029869 sec","job_status":"done","queue_duration_s":1.12997,"scheduling_latency_s":1.129999,"redis_calls":16,"redis_duration_s":0.003241,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001914,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034038,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029869,"completed_at":"2026-09-03T10:17:18.701Z","load_balancing_strategy":"primary","db_duration_s":0.000719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 256.001679] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa9ce11b8a80daafaef393b6","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa9ce11b8a80daafaef393b6: start","job_status":"start","queue_duration_s":1.161485,"scheduling_latency_s":1.161509}1291gitlab # [ 256.014286] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa9ce11b8a80daafaef393b6","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa9ce11b8a80daafaef393b6: done: 0.017923 sec","job_status":"done","queue_duration_s":1.161485,"scheduling_latency_s":1.161509,"redis_calls":13,"redis_duration_s":0.0016359999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017923,"completed_at":"2026-09-03T10:17:18.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 256.053633] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b566e725882961436b87871","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b566e725882961436b87871: start","job_status":"start","queue_duration_s":1.181263,"scheduling_latency_s":1.181287}1293gitlab # [ 256.066267] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b566e725882961436b87871","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b566e725882961436b87871: done: 0.021324 sec","job_status":"done","queue_duration_s":1.181263,"scheduling_latency_s":1.181287,"redis_calls":13,"redis_duration_s":0.002231,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021324,"completed_at":"2026-09-03T10:17:18.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 256.114718] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"260071ce64be8b4df91db2b3","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-260071ce64be8b4df91db2b3: start","job_status":"start","queue_duration_s":1.204076,"scheduling_latency_s":1.204101}1295gitlab # [ 256.127446] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"260071ce64be8b4df91db2b3","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-260071ce64be8b4df91db2b3: done: 0.018009 sec","job_status":"done","queue_duration_s":1.204076,"scheduling_latency_s":1.204101,"redis_calls":13,"redis_duration_s":0.001838,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018009,"completed_at":"2026-09-03T10:17:18.763Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 256.167277] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8907d11978c565f63bbbc7b8","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8907d11978c565f63bbbc7b8: start","job_status":"start","queue_duration_s":1.22345,"scheduling_latency_s":1.223473}1297gitlab # [ 256.180039] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8907d11978c565f63bbbc7b8","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8907d11978c565f63bbbc7b8: done: 0.019028 sec","job_status":"done","queue_duration_s":1.22345,"scheduling_latency_s":1.223473,"redis_calls":13,"redis_duration_s":0.001886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019028,"completed_at":"2026-09-03T10:17:18.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 256.219540] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8580c906ef46e2bf8be85e85","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8580c906ef46e2bf8be85e85: start","job_status":"start","queue_duration_s":1.244404,"scheduling_latency_s":1.244428}1299gitlab # [ 256.232260] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8580c906ef46e2bf8be85e85","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8580c906ef46e2bf8be85e85: done: 0.019265 sec","job_status":"done","queue_duration_s":1.244404,"scheduling_latency_s":1.244428,"redis_calls":13,"redis_duration_s":0.0029180000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019265,"completed_at":"2026-09-03T10:17:18.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 256.271665] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23fd9fb733d85ed0e0f396b3","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23fd9fb733d85ed0e0f396b3: start","job_status":"start","queue_duration_s":1.265237,"scheduling_latency_s":1.265262}1301gitlab # [ 256.284421] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23fd9fb733d85ed0e0f396b3","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23fd9fb733d85ed0e0f396b3: done: 0.019131 sec","job_status":"done","queue_duration_s":1.265237,"scheduling_latency_s":1.265262,"redis_calls":13,"redis_duration_s":0.002125,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001327,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019131,"completed_at":"2026-09-03T10:17:18.825Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 256.325042] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a789b2121b9368894f85ed1","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a789b2121b9368894f85ed1: start","job_status":"start","queue_duration_s":1.285849,"scheduling_latency_s":1.285874}1303gitlab # [ 256.337552] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a789b2121b9368894f85ed1","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a789b2121b9368894f85ed1: done: 0.018739 sec","job_status":"done","queue_duration_s":1.285849,"scheduling_latency_s":1.285874,"redis_calls":13,"redis_duration_s":0.002073,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001347,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018739,"completed_at":"2026-09-03T10:17:18.845Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 256.377018] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7401f53de669dc3ceacbec02","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7401f53de669dc3ceacbec02: start","job_status":"start","queue_duration_s":1.30698,"scheduling_latency_s":1.307005}1305gitlab # [ 256.389326] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7401f53de669dc3ceacbec02","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7401f53de669dc3ceacbec02: done: 0.018667 sec","job_status":"done","queue_duration_s":1.30698,"scheduling_latency_s":1.307005,"redis_calls":13,"redis_duration_s":0.001836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018667,"completed_at":"2026-09-03T10:17:18.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 256.428920] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0430887256ee1e3f7a612b17","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0430887256ee1e3f7a612b17: start","job_status":"start","queue_duration_s":1.327081,"scheduling_latency_s":1.327103}1307gitlab # [ 256.441454] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:18.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0430887256ee1e3f7a612b17","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0430887256ee1e3f7a612b17: done: 0.024124 sec","job_status":"done","queue_duration_s":1.327081,"scheduling_latency_s":1.327103,"redis_calls":13,"redis_duration_s":0.001928,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024124,"completed_at":"2026-09-03T10:17:18.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 256.480679] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c68b6dcfd51ce98c7749c4f","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c68b6dcfd51ce98c7749c4f: start","job_status":"start","queue_duration_s":1.547469,"scheduling_latency_s":1.547528}1309gitlab # [ 256.493198] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c68b6dcfd51ce98c7749c4f","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c68b6dcfd51ce98c7749c4f: done: 0.048292 sec","job_status":"done","queue_duration_s":1.547469,"scheduling_latency_s":1.547528,"redis_calls":13,"redis_duration_s":0.005669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002601,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.048292,"completed_at":"2026-09-03T10:17:19.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 256.533980] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c429d136b11b640ed786b6fc","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c429d136b11b640ed786b6fc: start","job_status":"start","queue_duration_s":1.599097,"scheduling_latency_s":1.599139}1311gitlab # [ 256.546439] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c429d136b11b640ed786b6fc","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c429d136b11b640ed786b6fc: done: 0.034444 sec","job_status":"done","queue_duration_s":1.599097,"scheduling_latency_s":1.599139,"redis_calls":13,"redis_duration_s":0.0035969999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001249,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002348,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034444,"completed_at":"2026-09-03T10:17:19.174Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 256.585897] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d089b432d86a8bf25e1bcb66","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d089b432d86a8bf25e1bcb66: start","job_status":"start","queue_duration_s":1.635563,"scheduling_latency_s":1.635588}1313gitlab # [ 256.598511] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d089b432d86a8bf25e1bcb66","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d089b432d86a8bf25e1bcb66: done: 0.026009 sec","job_status":"done","queue_duration_s":1.635563,"scheduling_latency_s":1.635588,"redis_calls":13,"redis_duration_s":0.003746,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026009,"completed_at":"2026-09-03T10:17:19.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 256.637685] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd1f591ba988cc9d7a2e72b6","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd1f591ba988cc9d7a2e72b6: start","job_status":"start","queue_duration_s":1.664372,"scheduling_latency_s":1.664393}1315gitlab # [ 256.650157] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd1f591ba988cc9d7a2e72b6","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd1f591ba988cc9d7a2e72b6: done: 0.023684 sec","job_status":"done","queue_duration_s":1.664372,"scheduling_latency_s":1.664393,"redis_calls":13,"redis_duration_s":0.003231,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023684,"completed_at":"2026-09-03T10:17:19.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 256.689509] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78b0111e8ce38181c003cd35","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78b0111e8ce38181c003cd35: start","job_status":"start","queue_duration_s":1.689811,"scheduling_latency_s":1.689829}1317gitlab # [ 256.703304] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78b0111e8ce38181c003cd35","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78b0111e8ce38181c003cd35: done: 0.018208 sec","job_status":"done","queue_duration_s":1.689811,"scheduling_latency_s":1.689829,"redis_calls":13,"redis_duration_s":0.0036049999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002626,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018208,"completed_at":"2026-09-03T10:17:19.249Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 256.742950] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11a903a92e459355ce308256","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11a903a92e459355ce308256: start","job_status":"start","queue_duration_s":1.709475,"scheduling_latency_s":1.709491}1319gitlab # [ 256.755438] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11a903a92e459355ce308256","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11a903a92e459355ce308256: done: 0.017874 sec","job_status":"done","queue_duration_s":1.709475,"scheduling_latency_s":1.709491,"redis_calls":13,"redis_duration_s":0.002437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017874,"completed_at":"2026-09-03T10:17:19.268Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 256.795885] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71989dd4b4c59dbd56b0ab0c","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71989dd4b4c59dbd56b0ab0c: start","job_status":"start","queue_duration_s":1.728718,"scheduling_latency_s":1.728734}1321gitlab # [ 256.808431] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71989dd4b4c59dbd56b0ab0c","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71989dd4b4c59dbd56b0ab0c: done: 0.015207 sec","job_status":"done","queue_duration_s":1.728718,"scheduling_latency_s":1.728734,"redis_calls":13,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015207,"completed_at":"2026-09-03T10:17:19.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 256.848383] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1edfbce7edb3f306db32a759","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1edfbce7edb3f306db32a759: start","job_status":"start","queue_duration_s":1.745856,"scheduling_latency_s":1.745872}1323gitlab # [ 256.861110] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1edfbce7edb3f306db32a759","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1edfbce7edb3f306db32a759: done: 0.015419 sec","job_status":"done","queue_duration_s":1.745856,"scheduling_latency_s":1.745872,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015419,"completed_at":"2026-09-03T10:17:19.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 256.900899] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"493e6c35049ec53c3c748f39","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-493e6c35049ec53c3c748f39: start","job_status":"start","queue_duration_s":1.762356,"scheduling_latency_s":1.762371}1325gitlab # [ 256.914771] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"493e6c35049ec53c3c748f39","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-493e6c35049ec53c3c748f39: done: 0.015222 sec","job_status":"done","queue_duration_s":1.762356,"scheduling_latency_s":1.762371,"redis_calls":13,"redis_duration_s":0.0017389999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015222,"completed_at":"2026-09-03T10:17:19.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 256.955132] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"534f745d59b06a366cab43fe","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-534f745d59b06a366cab43fe: start","job_status":"start","queue_duration_s":1.778683,"scheduling_latency_s":1.778698}1327gitlab # [ 256.967771] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"534f745d59b06a366cab43fe","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-534f745d59b06a366cab43fe: done: 0.014349 sec","job_status":"done","queue_duration_s":1.778683,"scheduling_latency_s":1.778698,"redis_calls":13,"redis_duration_s":0.0014199999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014349,"completed_at":"2026-09-03T10:17:19.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 257.007690] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baeaecd14e9d4e1e17321d54","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baeaecd14e9d4e1e17321d54: start","job_status":"start","queue_duration_s":1.794082,"scheduling_latency_s":1.794097}1329gitlab # [ 257.020362] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baeaecd14e9d4e1e17321d54","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baeaecd14e9d4e1e17321d54: done: 0.014853 sec","job_status":"done","queue_duration_s":1.794082,"scheduling_latency_s":1.794097,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014853,"completed_at":"2026-09-03T10:17:19.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 257.059652] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e4ff8217724386e497eb9b1","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e4ff8217724386e497eb9b1: start","job_status":"start","queue_duration_s":1.810081,"scheduling_latency_s":1.810097}1331gitlab # [ 257.072320] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e4ff8217724386e497eb9b1","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e4ff8217724386e497eb9b1: done: 0.014431 sec","job_status":"done","queue_duration_s":1.810081,"scheduling_latency_s":1.810097,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014431,"completed_at":"2026-09-03T10:17:19.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 257.111640] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01e26905166df718244a93bd","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01e26905166df718244a93bd: start","job_status":"start","queue_duration_s":1.825558,"scheduling_latency_s":1.825574}1333gitlab # [ 257.125444] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01e26905166df718244a93bd","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01e26905166df718244a93bd: done: 0.014545 sec","job_status":"done","queue_duration_s":1.825558,"scheduling_latency_s":1.825574,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014545,"completed_at":"2026-09-03T10:17:19.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 257.164966] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5731b00ae06e5bdc6c91777c","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5731b00ae06e5bdc6c91777c: start","job_status":"start","queue_duration_s":1.84119,"scheduling_latency_s":1.841205}1335gitlab # [ 257.177511] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5731b00ae06e5bdc6c91777c","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5731b00ae06e5bdc6c91777c: done: 0.014604 sec","job_status":"done","queue_duration_s":1.84119,"scheduling_latency_s":1.841205,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014604,"completed_at":"2026-09-03T10:17:19.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 257.216743] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb1b774c6189a10baf26886c","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb1b774c6189a10baf26886c: start","job_status":"start","queue_duration_s":1.856902,"scheduling_latency_s":1.856919}1337gitlab # [ 257.229053] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb1b774c6189a10baf26886c","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb1b774c6189a10baf26886c: done: 0.014804 sec","job_status":"done","queue_duration_s":1.856902,"scheduling_latency_s":1.856919,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014804,"completed_at":"2026-09-03T10:17:19.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 257.267778] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d66bed72a8b12b710bfa56da","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d66bed72a8b12b710bfa56da: start","job_status":"start","queue_duration_s":1.872778,"scheduling_latency_s":1.872794}1339gitlab # [ 257.280328] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d66bed72a8b12b710bfa56da","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d66bed72a8b12b710bfa56da: done: 0.014316 sec","job_status":"done","queue_duration_s":1.872778,"scheduling_latency_s":1.872794,"redis_calls":13,"redis_duration_s":0.001805,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001276,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014316,"completed_at":"2026-09-03T10:17:19.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 257.319675] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b99ff4ad7e81c49667a29a72","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b99ff4ad7e81c49667a29a72: start","job_status":"start","queue_duration_s":1.888148,"scheduling_latency_s":1.888164}1341gitlab # [ 257.333352] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b99ff4ad7e81c49667a29a72","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b99ff4ad7e81c49667a29a72: done: 0.01442 sec","job_status":"done","queue_duration_s":1.888148,"scheduling_latency_s":1.888164,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01442,"completed_at":"2026-09-03T10:17:19.443Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 257.372379] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0de53b52e7481a1b4a9b49e9","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0de53b52e7481a1b4a9b49e9: start","job_status":"start","queue_duration_s":1.903623,"scheduling_latency_s":1.903639}1343gitlab # [ 257.384716] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0de53b52e7481a1b4a9b49e9","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0de53b52e7481a1b4a9b49e9: done: 0.014283 sec","job_status":"done","queue_duration_s":1.903623,"scheduling_latency_s":1.903639,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014283,"completed_at":"2026-09-03T10:17:19.459Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 257.423653] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fde5ecab0ce1d58bf71bed8d","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fde5ecab0ce1d58bf71bed8d: start","job_status":"start","queue_duration_s":1.919432,"scheduling_latency_s":1.919449}1345gitlab # [ 257.436083] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fde5ecab0ce1d58bf71bed8d","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fde5ecab0ce1d58bf71bed8d: done: 0.014313 sec","job_status":"done","queue_duration_s":1.919432,"scheduling_latency_s":1.919449,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014313,"completed_at":"2026-09-03T10:17:19.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 257.475637] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9922b4b709ae718f30e57f2","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9922b4b709ae718f30e57f2: start","job_status":"start","queue_duration_s":1.934775,"scheduling_latency_s":1.934789}1347gitlab # [ 257.488244] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9922b4b709ae718f30e57f2","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9922b4b709ae718f30e57f2: done: 0.017591 sec","job_status":"done","queue_duration_s":1.934775,"scheduling_latency_s":1.934789,"redis_calls":13,"redis_duration_s":0.002112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017591,"completed_at":"2026-09-03T10:17:19.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 257.528449] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de2a7c5bcc905bbd0c73adfa","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de2a7c5bcc905bbd0c73adfa: start","job_status":"start","queue_duration_s":1.954955,"scheduling_latency_s":1.954999}1349gitlab # [ 257.540784] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de2a7c5bcc905bbd0c73adfa","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de2a7c5bcc905bbd0c73adfa: done: 0.037844 sec","job_status":"done","queue_duration_s":1.954955,"scheduling_latency_s":1.954999,"redis_calls":13,"redis_duration_s":0.003288,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037844,"completed_at":"2026-09-03T10:17:19.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 257.581243] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af8a84a064d04030a06adbf1","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af8a84a064d04030a06adbf1: start","job_status":"start","queue_duration_s":1.995347,"scheduling_latency_s":1.995383}1351gitlab # [ 257.594010] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af8a84a064d04030a06adbf1","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af8a84a064d04030a06adbf1: done: 0.034912 sec","job_status":"done","queue_duration_s":1.995347,"scheduling_latency_s":1.995383,"redis_calls":13,"redis_duration_s":0.00348,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034912,"completed_at":"2026-09-03T10:17:19.571Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 257.634453] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f438f77626698e20493f7061","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f438f77626698e20493f7061: start","job_status":"start","queue_duration_s":2.034905,"scheduling_latency_s":2.034948}1353gitlab # [ 257.647230] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f438f77626698e20493f7061","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f438f77626698e20493f7061: done: 0.036588 sec","job_status":"done","queue_duration_s":2.034905,"scheduling_latency_s":2.034948,"redis_calls":13,"redis_duration_s":0.004259,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00153,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002729,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036588,"completed_at":"2026-09-03T10:17:19.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 257.687664] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad6f404c90f7c4e4fc6fc8f7","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad6f404c90f7c4e4fc6fc8f7: start","job_status":"start","queue_duration_s":2.073789,"scheduling_latency_s":2.073818}1355gitlab # [ 257.700726] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad6f404c90f7c4e4fc6fc8f7","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad6f404c90f7c4e4fc6fc8f7: done: 0.029855 sec","job_status":"done","queue_duration_s":2.073789,"scheduling_latency_s":2.073818,"redis_calls":13,"redis_duration_s":0.007119,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005793,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029855,"completed_at":"2026-09-03T10:17:19.644Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 257.741716] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"304d2d966545b47eeec56829","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-304d2d966545b47eeec56829: start","job_status":"start","queue_duration_s":2.107818,"scheduling_latency_s":2.107848}1357gitlab # [ 257.754527] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"304d2d966545b47eeec56829","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-304d2d966545b47eeec56829: done: 0.026401 sec","job_status":"done","queue_duration_s":2.107818,"scheduling_latency_s":2.107848,"redis_calls":13,"redis_duration_s":0.003431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002288,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026401,"completed_at":"2026-09-03T10:17:19.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 257.793908] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dadb158db4d463ac8103af6","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dadb158db4d463ac8103af6: start","job_status":"start","queue_duration_s":2.136191,"scheduling_latency_s":2.136219}1359gitlab # [ 257.806726] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2dadb158db4d463ac8103af6","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2dadb158db4d463ac8103af6: done: 0.021452 sec","job_status":"done","queue_duration_s":2.136191,"scheduling_latency_s":2.136219,"redis_calls":13,"redis_duration_s":0.003675,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00261,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021452,"completed_at":"2026-09-03T10:17:19.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 257.846662] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0b43cf2571b41c4d1c93dd2","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0b43cf2571b41c4d1c93dd2: start","job_status":"start","queue_duration_s":2.159631,"scheduling_latency_s":2.159649}1361gitlab # [ 257.883868] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0b43cf2571b41c4d1c93dd2","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0b43cf2571b41c4d1c93dd2: done: 0.019383 sec","job_status":"done","queue_duration_s":2.159631,"scheduling_latency_s":2.159649,"redis_calls":13,"redis_duration_s":0.0030230000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019383,"completed_at":"2026-09-03T10:17:19.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 257.961800] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8da951e1f442015d0477f391","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8da951e1f442015d0477f391: start","job_status":"start","queue_duration_s":2.180606,"scheduling_latency_s":2.180624}1363gitlab # [ 257.981500] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8da951e1f442015d0477f391","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8da951e1f442015d0477f391: done: 0.017547 sec","job_status":"done","queue_duration_s":2.180606,"scheduling_latency_s":2.180624,"redis_calls":13,"redis_duration_s":0.0028,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017547,"completed_at":"2026-09-03T10:17:19.739Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 258.026961] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57a4d182b6e92deb14b85a47","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57a4d182b6e92deb14b85a47: start","job_status":"start","queue_duration_s":2.199678,"scheduling_latency_s":2.199693}1365gitlab # [ 258.039540] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57a4d182b6e92deb14b85a47","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57a4d182b6e92deb14b85a47: done: 0.015835 sec","job_status":"done","queue_duration_s":2.199678,"scheduling_latency_s":2.199693,"redis_calls":13,"redis_duration_s":0.002608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015835,"completed_at":"2026-09-03T10:17:19.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 258.079321] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41a0ee29fa39ca02ed5665fc","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41a0ee29fa39ca02ed5665fc: start","job_status":"start","queue_duration_s":2.21694,"scheduling_latency_s":2.216956}1367gitlab # [ 258.092084] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41a0ee29fa39ca02ed5665fc","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41a0ee29fa39ca02ed5665fc: done: 0.01573 sec","job_status":"done","queue_duration_s":2.21694,"scheduling_latency_s":2.216956,"redis_calls":13,"redis_duration_s":0.002706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001767,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01573,"completed_at":"2026-09-03T10:17:19.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 258.130993] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f17285959994ec6c0f413bd","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f17285959994ec6c0f413bd: start","job_status":"start","queue_duration_s":2.234649,"scheduling_latency_s":2.234695}1369gitlab # [ 258.144660] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f17285959994ec6c0f413bd","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f17285959994ec6c0f413bd: done: 0.037455 sec","job_status":"done","queue_duration_s":2.234649,"scheduling_latency_s":2.234695,"redis_calls":13,"redis_duration_s":0.003671,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037455,"completed_at":"2026-09-03T10:17:19.813Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 258.184072] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d8343415270cf66818feb68","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d8343415270cf66818feb68: start","job_status":"start","queue_duration_s":2.274729,"scheduling_latency_s":2.274764}1371gitlab # [ 258.196395] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d8343415270cf66818feb68","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d8343415270cf66818feb68: done: 0.028237 sec","job_status":"done","queue_duration_s":2.274729,"scheduling_latency_s":2.274764,"redis_calls":13,"redis_duration_s":0.003267,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028237,"completed_at":"2026-09-03T10:17:19.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 258.235933] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21f63abda5294a79e8ee5371","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21f63abda5294a79e8ee5371: start","job_status":"start","queue_duration_s":2.304834,"scheduling_latency_s":2.304857}1373gitlab # [ 258.248721] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21f63abda5294a79e8ee5371","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21f63abda5294a79e8ee5371: done: 0.020909 sec","job_status":"done","queue_duration_s":2.304834,"scheduling_latency_s":2.304857,"redis_calls":13,"redis_duration_s":0.00297,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020909,"completed_at":"2026-09-03T10:17:19.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 258.288116] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"803bd5f6c74ecfbbdb1ba4de","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-803bd5f6c74ecfbbdb1ba4de: start","job_status":"start","queue_duration_s":2.327847,"scheduling_latency_s":2.327867}1375gitlab # [ 258.300513] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"803bd5f6c74ecfbbdb1ba4de","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-803bd5f6c74ecfbbdb1ba4de: done: 0.018231 sec","job_status":"done","queue_duration_s":2.327847,"scheduling_latency_s":2.327867,"redis_calls":13,"redis_duration_s":0.002933,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018231,"completed_at":"2026-09-03T10:17:19.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 258.341092] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a040a71f62767e2504810e75","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a040a71f62767e2504810e75: start","job_status":"start","queue_duration_s":2.3475,"scheduling_latency_s":2.347515}1377gitlab # [ 258.353241] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a040a71f62767e2504810e75","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a040a71f62767e2504810e75: done: 0.015955 sec","job_status":"done","queue_duration_s":2.3475,"scheduling_latency_s":2.347515,"redis_calls":13,"redis_duration_s":0.002816,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015955,"completed_at":"2026-09-03T10:17:19.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 258.392620] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"337b46600063d9cd375cb3ff","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-337b46600063d9cd375cb3ff: start","job_status":"start","queue_duration_s":2.364736,"scheduling_latency_s":2.36475}1379gitlab # [ 258.405082] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"337b46600063d9cd375cb3ff","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-337b46600063d9cd375cb3ff: done: 0.015114 sec","job_status":"done","queue_duration_s":2.364736,"scheduling_latency_s":2.36475,"redis_calls":13,"redis_duration_s":0.002538,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001622,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015114,"completed_at":"2026-09-03T10:17:19.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 258.444254] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e1f5bd32a37a7093acbb832","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e1f5bd32a37a7093acbb832: start","job_status":"start","queue_duration_s":2.38124,"scheduling_latency_s":2.381254}1381gitlab # [ 258.456731] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e1f5bd32a37a7093acbb832","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e1f5bd32a37a7093acbb832: done: 0.015162 sec","job_status":"done","queue_duration_s":2.38124,"scheduling_latency_s":2.381254,"redis_calls":13,"redis_duration_s":0.002598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001692,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015162,"completed_at":"2026-09-03T10:17:19.937Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 258.496400] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15f7a3585108f8e0539bb9a5","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15f7a3585108f8e0539bb9a5: start","job_status":"start","queue_duration_s":2.397915,"scheduling_latency_s":2.39793}1383gitlab # [ 258.508562] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15f7a3585108f8e0539bb9a5","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15f7a3585108f8e0539bb9a5: done: 0.015369 sec","job_status":"done","queue_duration_s":2.397915,"scheduling_latency_s":2.39793,"redis_calls":13,"redis_duration_s":0.002694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015369,"completed_at":"2026-09-03T10:17:19.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 258.549742] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99ddbcf859959cf524854106","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99ddbcf859959cf524854106: start","job_status":"start","queue_duration_s":2.414618,"scheduling_latency_s":2.414632}1385gitlab # [ 258.562254] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99ddbcf859959cf524854106","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99ddbcf859959cf524854106: done: 0.015577 sec","job_status":"done","queue_duration_s":2.414618,"scheduling_latency_s":2.414632,"redis_calls":13,"redis_duration_s":0.002752,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001797,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015577,"completed_at":"2026-09-03T10:17:19.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 258.601729] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f417b99879aa59bf8202074","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f417b99879aa59bf8202074: start","job_status":"start","queue_duration_s":2.431507,"scheduling_latency_s":2.431522}1387gitlab # [ 258.614357] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:17:19.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f417b99879aa59bf8202074","created_at":"2026-09-03T10:17:17.038Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-55c3035ddf2142c6","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1c4217ed858c379d270baafa94309fac","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:17:17.541Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f417b99879aa59bf8202074: done: 0.01495 sec","job_status":"done","queue_duration_s":2.431507,"scheduling_latency_s":2.431522,"redis_calls":13,"redis_duration_s":0.002582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001669,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01495,"completed_at":"2026-09-03T10:17:19.987Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 284.690433] setup-gitlab-start[3114]: W, [2026-09-03T10:17:47.539106 #3114] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1389gitlab # [ 302.543694] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:05.391Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"039031c008b410d74835779a","created_at":"2026-09-03T10:18:05.371Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3a90c5e9bec74f567c0a3883294f7d9","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.387Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-039031c008b410d74835779a: start","job_status":"start","queue_duration_s":0.003759,"scheduling_latency_s":0.003818}1390gitlab # [ 303.984281] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.832Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"039031c008b410d74835779a","created_at":"2026-09-03T10:18:05.371Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3a90c5e9bec74f567c0a3883294f7d9","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.387Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-039031c008b410d74835779a: done: 1.441136 sec","job_status":"done","queue_duration_s":0.003759,"scheduling_latency_s":0.003818,"redis_calls":569,"redis_duration_s":0.737475,"redis_read_bytes":1456,"redis_write_bytes":49240,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.006744,"redis_feature_flag_read_bytes":896,"redis_feature_flag_write_bytes":480,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.729044,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.733439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.441136,"completed_at":"2026-09-03T10:18:06.832Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 303.985040] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3ab93806c4c949377e1174a0","created_at":"2026-09-03T10:18:05.465Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1fe287b671cf4ae23c0842fd0bc2f86","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.498Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3ab93806c4c949377e1174a0: start","job_status":"start","queue_duration_s":1.336465,"scheduling_latency_s":1.336493}1392gitlab # [ 304.085731] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3ab93806c4c949377e1174a0","created_at":"2026-09-03T10:18:05.465Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1fe287b671cf4ae23c0842fd0bc2f86","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.498Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3ab93806c4c949377e1174a0: done: 0.022353 sec","job_status":"done","queue_duration_s":1.336465,"scheduling_latency_s":1.336493,"redis_calls":8,"redis_duration_s":0.00164,"redis_read_bytes":133,"redis_write_bytes":730,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000201,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001027,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022353,"completed_at":"2026-09-03T10:18:06.857Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000834,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 304.145083] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"9bf5af5a547b170240414ed5","created_at":"2026-09-03T10:18:05.519Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86f7cbe8b9a0e0e8209c489812305ac4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.540Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-9bf5af5a547b170240414ed5: start","job_status":"start","queue_duration_s":1.318358,"scheduling_latency_s":1.318386}1394gitlab # [ 304.160112] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"9bf5af5a547b170240414ed5","created_at":"2026-09-03T10:18:05.519Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86f7cbe8b9a0e0e8209c489812305ac4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.540Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-9bf5af5a547b170240414ed5: done: 0.025482 sec","job_status":"done","queue_duration_s":1.318358,"scheduling_latency_s":1.318386,"redis_calls":12,"redis_duration_s":0.0022700000000000003,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000469,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029308,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.025482,"completed_at":"2026-09-03T10:18:06.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 304.205153] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.885Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a8123f46661cbff1de7e4ead","created_at":"2026-09-03T10:18:05.557Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cee07d9fa62e553d98cbf24c4539d1a9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.567Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a8123f46661cbff1de7e4ead: start","job_status":"start","queue_duration_s":1.31859,"scheduling_latency_s":1.318609}1396gitlab # [ 304.219627] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a8123f46661cbff1de7e4ead","created_at":"2026-09-03T10:18:05.557Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cee07d9fa62e553d98cbf24c4539d1a9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.567Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a8123f46661cbff1de7e4ead: done: 0.037849 sec","job_status":"done","queue_duration_s":1.31859,"scheduling_latency_s":1.318609,"redis_calls":9,"redis_duration_s":0.001436,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000706,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006013,"exclusive_lock_hold_duration_s":0.01334072,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.037849,"completed_at":"2026-09-03T10:18:06.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 304.262248] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b5cec3b36a1820ca4552f97a","created_at":"2026-09-03T10:18:05.587Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"596d205189c7fefadb880367351d9e25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.598Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b5cec3b36a1820ca4552f97a: start","job_status":"start","queue_duration_s":1.326458,"scheduling_latency_s":1.326475}1398gitlab # [ 304.278423] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.933Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1399gitlab # [ 304.281923] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b5cec3b36a1820ca4552f97a","created_at":"2026-09-03T10:18:05.587Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"596d205189c7fefadb880367351d9e25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.598Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b5cec3b36a1820ca4552f97a: done: 0.016439 sec","job_status":"done","queue_duration_s":1.326458,"scheduling_latency_s":1.326475,"redis_calls":7,"redis_duration_s":0.001098,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016439,"completed_at":"2026-09-03T10:18:06.941Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 304.322301] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9decbd706711e5c0360a0f67","created_at":"2026-09-03T10:18:05.632Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5ef4e5b832794c8e32bb9fc608a470c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.654Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9decbd706711e5c0360a0f67: start","job_status":"start","queue_duration_s":1.288165,"scheduling_latency_s":1.288182}1401gitlab # [ 304.336779] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9decbd706711e5c0360a0f67","created_at":"2026-09-03T10:18:05.632Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5ef4e5b832794c8e32bb9fc608a470c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.654Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9decbd706711e5c0360a0f67: done: 0.02716 sec","job_status":"done","queue_duration_s":1.288165,"scheduling_latency_s":1.288182,"redis_calls":12,"redis_duration_s":0.0018110000000000001,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000342,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000692,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005004,"exclusive_lock_hold_duration_s":0.0057307,"duration_s":0.02716,"completed_at":"2026-09-03T10:18:06.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 304.378967] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:06.971Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e3b816baddc75aa72eced053","created_at":"2026-09-03T10:18:05.696Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b54e328d79678481122b7958e46212d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.705Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e3b816baddc75aa72eced053: start","job_status":"start","queue_duration_s":1.265715,"scheduling_latency_s":1.265735}1403gitlab # [ 304.393043] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e3b816baddc75aa72eced053","created_at":"2026-09-03T10:18:05.696Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b54e328d79678481122b7958e46212d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.705Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e3b816baddc75aa72eced053: done: 0.241195 sec","job_status":"done","queue_duration_s":1.265715,"scheduling_latency_s":1.265735,"redis_calls":11,"redis_duration_s":0.00201,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000373,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000813,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.212021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052258,"exclusive_lock_hold_duration_s":0.22105376,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.241195,"completed_at":"2026-09-03T10:18:07.212Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 304.437232] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e5b2746201761d531548cbda","created_at":"2026-09-03T10:18:05.728Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f54a06fbbdaef4cabc5e91f34cff840b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:18:05.737Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e5b2746201761d531548cbda: start","job_status":"start","queue_duration_s":1.476077,"scheduling_latency_s":1.4761}1405gitlab # [ 304.450550] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.223Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1406gitlab # [ 304.453920] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e5b2746201761d531548cbda","created_at":"2026-09-03T10:18:05.728Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f54a06fbbdaef4cabc5e91f34cff840b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:18:05.737Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e5b2746201761d531548cbda: done: 0.015297 sec","job_status":"done","queue_duration_s":1.476077,"scheduling_latency_s":1.4761,"redis_calls":7,"redis_duration_s":0.001103,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000316,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015297,"completed_at":"2026-09-03T10:18:07.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 304.494439] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"143f6134c6e28cdfe3311edc","created_at":"2026-09-03T10:18:05.758Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4b67ac979802b47abec5bd9a27bf010","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:18:05.767Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-143f6134c6e28cdfe3311edc: start","job_status":"start","queue_duration_s":1.462867,"scheduling_latency_s":1.462886}1408gitlab # [ 304.507799] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.239Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1409gitlab # [ 304.511659] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.247Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"143f6134c6e28cdfe3311edc","created_at":"2026-09-03T10:18:05.758Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4b67ac979802b47abec5bd9a27bf010","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:18:05.767Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-143f6134c6e28cdfe3311edc: done: 0.016548 sec","job_status":"done","queue_duration_s":1.462867,"scheduling_latency_s":1.462886,"redis_calls":7,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016548,"completed_at":"2026-09-03T10:18:07.247Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 304.550927] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"59513d53285548835a889954","created_at":"2026-09-03T10:18:05.789Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.798Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-59513d53285548835a889954: start","job_status":"start","queue_duration_s":1.449926,"scheduling_latency_s":1.449943}1411gitlab # [ 304.564419] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.274Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"59513d53285548835a889954","created_at":"2026-09-03T10:18:05.789Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.798Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-59513d53285548835a889954: done: 0.026452 sec","job_status":"done","queue_duration_s":1.449926,"scheduling_latency_s":1.449943,"redis_calls":14,"redis_duration_s":0.001992,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000261,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026452,"completed_at":"2026-09-03T10:18:07.274Z","load_balancing_strategy":"primary","db_duration_s":0.000585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 304.605233] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8ae5fa651412cb6f2161870f","created_at":"2026-09-03T10:18:05.832Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c75a55eb3282887e74cfe0b6fe6a5d39","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.842Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8ae5fa651412cb6f2161870f: start","job_status":"start","queue_duration_s":1.434514,"scheduling_latency_s":1.434531}1413gitlab # [ 304.619557] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.284Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1414gitlab # [ 304.622886] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8ae5fa651412cb6f2161870f","created_at":"2026-09-03T10:18:05.832Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c75a55eb3282887e74cfe0b6fe6a5d39","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.842Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8ae5fa651412cb6f2161870f: done: 0.013759 sec","job_status":"done","queue_duration_s":1.434514,"scheduling_latency_s":1.434531,"redis_calls":7,"redis_duration_s":0.000985,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000301,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013759,"completed_at":"2026-09-03T10:18:07.290Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 304.663336] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.291Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b73c09e8f5a6cc35e82ebec6","created_at":"2026-09-03T10:18:05.868Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4249279a5de0015e76ca76721f911c7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.877Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b73c09e8f5a6cc35e82ebec6: start","job_status":"start","queue_duration_s":1.414515,"scheduling_latency_s":1.414532}1416gitlab # [ 304.678917] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.324Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b73c09e8f5a6cc35e82ebec6","created_at":"2026-09-03T10:18:05.868Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4249279a5de0015e76ca76721f911c7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.877Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b73c09e8f5a6cc35e82ebec6: done: 0.032783 sec","job_status":"done","queue_duration_s":1.414515,"scheduling_latency_s":1.414532,"redis_calls":7,"redis_duration_s":0.001618,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032783,"completed_at":"2026-09-03T10:18:07.324Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 304.719360] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3274505dc2c9decfa0bd2828","created_at":"2026-09-03T10:18:05.942Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.950Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3274505dc2c9decfa0bd2828: start","job_status":"start","queue_duration_s":1.377171,"scheduling_latency_s":1.377212}1418gitlab # [ 304.732893] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3274505dc2c9decfa0bd2828","created_at":"2026-09-03T10:18:05.942Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.950Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3274505dc2c9decfa0bd2828: done: 0.163009 sec","job_status":"done","queue_duration_s":1.377171,"scheduling_latency_s":1.377212,"redis_calls":60,"redis_duration_s":0.018634,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000978,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.01682,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.145353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.163009,"completed_at":"2026-09-03T10:18:07.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 304.773849] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c0308a057a2a9ff0603f325e","created_at":"2026-09-03T10:18:05.969Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c36bc08c2e47d07f6d2a5be5c409beb6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.978Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c0308a057a2a9ff0603f325e: start","job_status":"start","queue_duration_s":1.513372,"scheduling_latency_s":1.513399}1420gitlab # [ 304.788440] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.502Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1421gitlab # [ 304.792089] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c0308a057a2a9ff0603f325e","created_at":"2026-09-03T10:18:05.969Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c36bc08c2e47d07f6d2a5be5c409beb6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:05.978Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c0308a057a2a9ff0603f325e: done: 0.016559 sec","job_status":"done","queue_duration_s":1.513372,"scheduling_latency_s":1.513399,"redis_calls":7,"redis_duration_s":0.001822,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016559,"completed_at":"2026-09-03T10:18:07.508Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 304.832887] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"268766bd02f5ec2573b6d60b","created_at":"2026-09-03T10:18:06.011Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05b635fd23b768c91bca9d32eb5e5ffa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.020Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-268766bd02f5ec2573b6d60b: start","job_status":"start","queue_duration_s":1.489954,"scheduling_latency_s":1.489982}1423gitlab # [ 304.846367] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"268766bd02f5ec2573b6d60b","created_at":"2026-09-03T10:18:06.011Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05b635fd23b768c91bca9d32eb5e5ffa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.020Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-268766bd02f5ec2573b6d60b: done: 0.037301 sec","job_status":"done","queue_duration_s":1.489954,"scheduling_latency_s":1.489982,"redis_calls":9,"redis_duration_s":0.0037879999999999997,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001658,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00123116,"exclusive_lock_hold_duration_s":0.0076353,"duration_s":0.037301,"completed_at":"2026-09-03T10:18:07.547Z","load_balancing_strategy":"primary","db_duration_s":0.001391,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 304.887594] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.552Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"070ed308ccfacc3a6a499bdc","created_at":"2026-09-03T10:18:06.131Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"421a5b02ca1fbb0e7924544fff2670c5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.140Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-070ed308ccfacc3a6a499bdc: start","job_status":"start","queue_duration_s":1.411104,"scheduling_latency_s":1.411158}1425gitlab # [ 304.902292] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"070ed308ccfacc3a6a499bdc","created_at":"2026-09-03T10:18:06.131Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"421a5b02ca1fbb0e7924544fff2670c5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.140Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-070ed308ccfacc3a6a499bdc: done: 0.086148 sec","job_status":"done","queue_duration_s":1.411104,"scheduling_latency_s":1.411158,"redis_calls":9,"redis_duration_s":0.004255,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00135032,"exclusive_lock_hold_duration_s":0.04493282,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.086148,"completed_at":"2026-09-03T10:18:07.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 304.945049] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"993ea886e9ebb339b313e28b","created_at":"2026-09-03T10:18:06.174Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"870b03803938b0a7c9e52827b708d848","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.202Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-993ea886e9ebb339b313e28b: start","job_status":"start","queue_duration_s":1.438696,"scheduling_latency_s":1.438734}1427gitlab # [ 304.959585] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.691Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"993ea886e9ebb339b313e28b","created_at":"2026-09-03T10:18:06.174Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"870b03803938b0a7c9e52827b708d848","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.202Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-993ea886e9ebb339b313e28b: done: 0.050271 sec","job_status":"done","queue_duration_s":1.438696,"scheduling_latency_s":1.438734,"redis_calls":10,"redis_duration_s":0.003598,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000655,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002112,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006057,"duration_s":0.050271,"completed_at":"2026-09-03T10:18:07.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 305.001823] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"035a96c6cb89806f02613f8b","created_at":"2026-09-03T10:18:06.227Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2138d2eda98970d1a7911e27a4c5a84","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.243Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-035a96c6cb89806f02613f8b: start","job_status":"start","queue_duration_s":1.449587,"scheduling_latency_s":1.449621}1429gitlab # [ 305.016399] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"035a96c6cb89806f02613f8b","created_at":"2026-09-03T10:18:06.227Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2138d2eda98970d1a7911e27a4c5a84","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.243Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-035a96c6cb89806f02613f8b: done: 0.036424 sec","job_status":"done","queue_duration_s":1.449587,"scheduling_latency_s":1.449621,"redis_calls":10,"redis_duration_s":0.005086,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003282,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001804,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0074773,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.036424,"completed_at":"2026-09-03T10:18:07.729Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003088,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1430gitlab # [ 305.058894] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.732Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"717a4fc74ce8aca9589ada39","created_at":"2026-09-03T10:18:06.269Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c49de907c95b57c18362399941a14ca3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.279Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-717a4fc74ce8aca9589ada39: start","job_status":"start","queue_duration_s":1.453377,"scheduling_latency_s":1.453411}1431gitlab # [ 305.073369] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.744Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1432gitlab # [ 305.078048] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"717a4fc74ce8aca9589ada39","created_at":"2026-09-03T10:18:06.269Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c49de907c95b57c18362399941a14ca3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.279Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-717a4fc74ce8aca9589ada39: done: 0.022159 sec","job_status":"done","queue_duration_s":1.453377,"scheduling_latency_s":1.453411,"redis_calls":7,"redis_duration_s":0.00292,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022159,"completed_at":"2026-09-03T10:18:07.754Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 305.118257] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.757Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"8b0a4a559a29d08b33021ad1","created_at":"2026-09-03T10:18:06.309Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f0a23f7902ee86014420c1abaf876ea","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.332Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-8b0a4a559a29d08b33021ad1: start","job_status":"start","queue_duration_s":1.425095,"scheduling_latency_s":1.425125}1434gitlab # [ 305.131243] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.786Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"8b0a4a559a29d08b33021ad1","created_at":"2026-09-03T10:18:06.309Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f0a23f7902ee86014420c1abaf876ea","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.332Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-8b0a4a559a29d08b33021ad1: done: 0.029709 sec","job_status":"done","queue_duration_s":1.425095,"scheduling_latency_s":1.425125,"redis_calls":12,"redis_duration_s":0.004097,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000761,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000981,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041332,"duration_s":0.029709,"completed_at":"2026-09-03T10:18:07.786Z","load_balancing_strategy":"primary","db_duration_s":0.00096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 305.173391] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.788Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f3a054e5701b8252c6c4b996","created_at":"2026-09-03T10:18:06.354Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf3c3d46bb65035920c89387f93d80d4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.364Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f3a054e5701b8252c6c4b996: start","job_status":"start","queue_duration_s":1.424384,"scheduling_latency_s":1.424424}1436gitlab # [ 305.187488] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.809Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f3a054e5701b8252c6c4b996","created_at":"2026-09-03T10:18:06.354Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf3c3d46bb65035920c89387f93d80d4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.364Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f3a054e5701b8252c6c4b996: done: 0.02107 sec","job_status":"done","queue_duration_s":1.424384,"scheduling_latency_s":1.424424,"redis_calls":9,"redis_duration_s":0.002776,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001355,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00077084,"exclusive_lock_hold_duration_s":0.00329434,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.02107,"completed_at":"2026-09-03T10:18:07.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 305.229025] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"cc0a12f161e65a6c39a5f5e4","created_at":"2026-09-03T10:18:06.388Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99f4159830ced51b8fd0f9ebaebf934a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:18:06.397Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-cc0a12f161e65a6c39a5f5e4: start","job_status":"start","queue_duration_s":1.413895,"scheduling_latency_s":1.413912}1438gitlab # [ 305.241920] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"cc0a12f161e65a6c39a5f5e4","created_at":"2026-09-03T10:18:06.388Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99f4159830ced51b8fd0f9ebaebf934a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:18:06.397Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-cc0a12f161e65a6c39a5f5e4: done: 0.024034 sec","job_status":"done","queue_duration_s":1.413895,"scheduling_latency_s":1.413912,"redis_calls":8,"redis_duration_s":0.002178,"redis_read_bytes":133,"redis_write_bytes":811,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000221,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001455,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024034,"completed_at":"2026-09-03T10:18:07.835Z","load_balancing_strategy":"primary","db_duration_s":0.001992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 305.282266] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8068085f8fdd44c0798a8ee8","created_at":"2026-09-03T10:18:06.421Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84dc857d402960390484441f77aa37ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.432Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8068085f8fdd44c0798a8ee8: start","job_status":"start","queue_duration_s":1.404396,"scheduling_latency_s":1.404415}1440gitlab # [ 305.298320] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.851Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8068085f8fdd44c0798a8ee8","created_at":"2026-09-03T10:18:06.421Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84dc857d402960390484441f77aa37ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.432Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8068085f8fdd44c0798a8ee8: done: 0.014066 sec","job_status":"done","queue_duration_s":1.404396,"scheduling_latency_s":1.404415,"redis_calls":7,"redis_duration_s":0.00172,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014066,"completed_at":"2026-09-03T10:18:07.851Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 305.338428] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"be07bf397df7a685e5495ecb","created_at":"2026-09-03T10:18:06.462Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39eff741ca4503d7f0f90e036ec7ac1d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.473Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-be07bf397df7a685e5495ecb: start","job_status":"start","queue_duration_s":1.378974,"scheduling_latency_s":1.37899}1442gitlab # [ 305.352341] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.872Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"be07bf397df7a685e5495ecb","created_at":"2026-09-03T10:18:06.462Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39eff741ca4503d7f0f90e036ec7ac1d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.473Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-be07bf397df7a685e5495ecb: done: 0.019829 sec","job_status":"done","queue_duration_s":1.378974,"scheduling_latency_s":1.37899,"redis_calls":7,"redis_duration_s":0.001987,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000548,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019829,"completed_at":"2026-09-03T10:18:07.872Z","load_balancing_strategy":"primary","db_duration_s":0.004273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 305.392604] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.874Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d32daec10fb13c2502388efc","created_at":"2026-09-03T10:18:06.488Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"740a848b6f192cf84bcdb172b29216d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.507Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d32daec10fb13c2502388efc: start","job_status":"start","queue_duration_s":1.366569,"scheduling_latency_s":1.366586}1444gitlab # [ 305.407042] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.887Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d32daec10fb13c2502388efc","created_at":"2026-09-03T10:18:06.488Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-b0cb4c3f3b294d07","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"740a848b6f192cf84bcdb172b29216d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:06.507Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d32daec10fb13c2502388efc: done: 0.013631 sec","job_status":"done","queue_duration_s":1.366569,"scheduling_latency_s":1.366586,"redis_calls":7,"redis_duration_s":0.0016870000000000001,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013631,"completed_at":"2026-09-03T10:18:07.887Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 305.447855] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.889Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7e4974c880056799ecb072b6","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7e4974c880056799ecb072b6: start","job_status":"start","queue_duration_s":0.620142,"scheduling_latency_s":0.620157}1446gitlab # [ 305.460435] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.908Z","message":"No existing merge request to be cleaned up."}1447gitlab # [ 305.462077] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.917Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7e4974c880056799ecb072b6","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7e4974c880056799ecb072b6: done: 0.028454 sec","job_status":"done","queue_duration_s":0.620142,"scheduling_latency_s":0.620157,"redis_calls":10,"redis_duration_s":0.002771,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001661,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028454,"completed_at":"2026-09-03T10:18:07.917Z","load_balancing_strategy":"primary","db_duration_s":0.002951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 305.501266] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.919Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8eb3fc51f0037a0b1dda28da","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8eb3fc51f0037a0b1dda28da: start","job_status":"start","queue_duration_s":0.650047,"scheduling_latency_s":0.650062}1449gitlab # [ 305.513790] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.930Z","message":"No existing merge request to be cleaned up."}1450gitlab # [ 305.515535] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.939Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8eb3fc51f0037a0b1dda28da","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8eb3fc51f0037a0b1dda28da: done: 0.020397 sec","job_status":"done","queue_duration_s":0.650047,"scheduling_latency_s":0.650062,"redis_calls":10,"redis_duration_s":0.0024939999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001475,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020397,"completed_at":"2026-09-03T10:18:07.939Z","load_balancing_strategy":"primary","db_duration_s":0.002988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 305.553769] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.941Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3518ee9751a2ac60a0add441","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3518ee9751a2ac60a0add441: start","job_status":"start","queue_duration_s":0.671907,"scheduling_latency_s":0.671923}1452gitlab # [ 305.566201] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.952Z","message":"No existing merge request to be cleaned up."}1453gitlab # [ 305.568150] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.962Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3518ee9751a2ac60a0add441","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3518ee9751a2ac60a0add441: done: 0.021032 sec","job_status":"done","queue_duration_s":0.671907,"scheduling_latency_s":0.671923,"redis_calls":10,"redis_duration_s":0.002619,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001537,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021032,"completed_at":"2026-09-03T10:18:07.962Z","load_balancing_strategy":"primary","db_duration_s":0.00338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 305.606425] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.963Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2ddbd9361498202706cec738","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2ddbd9361498202706cec738: start","job_status":"start","queue_duration_s":0.694327,"scheduling_latency_s":0.694342}1455gitlab # [ 305.618667] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.974Z","message":"No existing merge request to be cleaned up."}1456gitlab # [ 305.620396] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.983Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2ddbd9361498202706cec738","created_at":"2026-09-03T10:18:07.258Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-189ba76dbfed4e35","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d7e677dbfd4fb0621ed6c2524a4d80e1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.269Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2ddbd9361498202706cec738: done: 0.02043 sec","job_status":"done","queue_duration_s":0.694327,"scheduling_latency_s":0.694342,"redis_calls":10,"redis_duration_s":0.002548,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001485,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02043,"completed_at":"2026-09-03T10:18:07.983Z","load_balancing_strategy":"primary","db_duration_s":0.002455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 305.658685] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:07.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76506a29dd6f0f4f43ed3540","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76506a29dd6f0f4f43ed3540: start","job_status":"start","queue_duration_s":0.504522,"scheduling_latency_s":0.504537}1458gitlab # [ 305.671218] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76506a29dd6f0f4f43ed3540","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76506a29dd6f0f4f43ed3540: done: 0.015838 sec","job_status":"done","queue_duration_s":0.504522,"scheduling_latency_s":0.504537,"redis_calls":13,"redis_duration_s":0.002844,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015838,"completed_at":"2026-09-03T10:18:08.001Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 305.711319] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"585813008fd4801555440bac","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-585813008fd4801555440bac: start","job_status":"start","queue_duration_s":0.521703,"scheduling_latency_s":0.521719}1460gitlab # [ 305.723701] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"585813008fd4801555440bac","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-585813008fd4801555440bac: done: 0.015229 sec","job_status":"done","queue_duration_s":0.521703,"scheduling_latency_s":0.521719,"redis_calls":13,"redis_duration_s":0.0027359999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015229,"completed_at":"2026-09-03T10:18:08.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 305.766103] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab44b26ad90da5bdcb3eae6c","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab44b26ad90da5bdcb3eae6c: start","job_status":"start","queue_duration_s":0.538219,"scheduling_latency_s":0.538233}1462gitlab # [ 305.778222] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab44b26ad90da5bdcb3eae6c","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab44b26ad90da5bdcb3eae6c: done: 0.014851 sec","job_status":"done","queue_duration_s":0.538219,"scheduling_latency_s":0.538233,"redis_calls":13,"redis_duration_s":0.0026449999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001715,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014851,"completed_at":"2026-09-03T10:18:08.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 305.817155] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1330303466f1ff0d14700dcc","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1330303466f1ff0d14700dcc: start","job_status":"start","queue_duration_s":0.554807,"scheduling_latency_s":0.55482}1464gitlab # [ 305.829375] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1330303466f1ff0d14700dcc","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1330303466f1ff0d14700dcc: done: 0.014947 sec","job_status":"done","queue_duration_s":0.554807,"scheduling_latency_s":0.55482,"redis_calls":13,"redis_duration_s":0.002646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001742,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014947,"completed_at":"2026-09-03T10:18:08.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 305.867631] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aed7d907847aab8286b66e07","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aed7d907847aab8286b66e07: start","job_status":"start","queue_duration_s":0.57183,"scheduling_latency_s":0.571846}1466gitlab # [ 305.880030] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aed7d907847aab8286b66e07","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aed7d907847aab8286b66e07: done: 0.015339 sec","job_status":"done","queue_duration_s":0.57183,"scheduling_latency_s":0.571846,"redis_calls":13,"redis_duration_s":0.002801,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015339,"completed_at":"2026-09-03T10:18:08.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 305.919526] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"331cd89284382c2e8addaee1","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-331cd89284382c2e8addaee1: start","job_status":"start","queue_duration_s":0.588573,"scheduling_latency_s":0.588587}1468gitlab # [ 305.932088] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"331cd89284382c2e8addaee1","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-331cd89284382c2e8addaee1: done: 0.015371 sec","job_status":"done","queue_duration_s":0.588573,"scheduling_latency_s":0.588587,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015371,"completed_at":"2026-09-03T10:18:08.084Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 305.970443] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e666057d06f238d88f5198d","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e666057d06f238d88f5198d: start","job_status":"start","queue_duration_s":0.605195,"scheduling_latency_s":0.605209}1470gitlab # [ 305.982557] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e666057d06f238d88f5198d","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e666057d06f238d88f5198d: done: 0.014933 sec","job_status":"done","queue_duration_s":0.605195,"scheduling_latency_s":0.605209,"redis_calls":13,"redis_duration_s":0.002443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001528,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014933,"completed_at":"2026-09-03T10:18:08.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 306.021135] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12a3e408693f95711ae6d325","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12a3e408693f95711ae6d325: start","job_status":"start","queue_duration_s":0.621407,"scheduling_latency_s":0.621421}1472gitlab # [ 306.033495] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12a3e408693f95711ae6d325","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12a3e408693f95711ae6d325: done: 0.014844 sec","job_status":"done","queue_duration_s":0.621407,"scheduling_latency_s":0.621421,"redis_calls":13,"redis_duration_s":0.002467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001549,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014844,"completed_at":"2026-09-03T10:18:08.116Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 306.074881] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10637f3aef489f3e27f33741","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10637f3aef489f3e27f33741: start","job_status":"start","queue_duration_s":0.637493,"scheduling_latency_s":0.637509}1474gitlab # [ 306.087623] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10637f3aef489f3e27f33741","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10637f3aef489f3e27f33741: done: 0.014877 sec","job_status":"done","queue_duration_s":0.637493,"scheduling_latency_s":0.637509,"redis_calls":13,"redis_duration_s":0.002827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014877,"completed_at":"2026-09-03T10:18:08.133Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 306.128108] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"758a33689e244dae7aed4802","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-758a33689e244dae7aed4802: start","job_status":"start","queue_duration_s":0.653764,"scheduling_latency_s":0.65378}1476gitlab # [ 306.140481] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"758a33689e244dae7aed4802","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-758a33689e244dae7aed4802: done: 0.014776 sec","job_status":"done","queue_duration_s":0.653764,"scheduling_latency_s":0.65378,"redis_calls":13,"redis_duration_s":0.002692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001771,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014776,"completed_at":"2026-09-03T10:18:08.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 306.179387] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8dbb5b8e9b713352d7e76e3","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8dbb5b8e9b713352d7e76e3: start","job_status":"start","queue_duration_s":0.669916,"scheduling_latency_s":0.669984}1478gitlab # [ 306.192036] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8dbb5b8e9b713352d7e76e3","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8dbb5b8e9b713352d7e76e3: done: 0.015169 sec","job_status":"done","queue_duration_s":0.669916,"scheduling_latency_s":0.669984,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015169,"completed_at":"2026-09-03T10:18:08.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 306.231288] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06c26dc63fb73e5278b78971","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06c26dc63fb73e5278b78971: start","job_status":"start","queue_duration_s":0.686452,"scheduling_latency_s":0.686466}1480gitlab # [ 306.243720] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06c26dc63fb73e5278b78971","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06c26dc63fb73e5278b78971: done: 0.014567 sec","job_status":"done","queue_duration_s":0.686452,"scheduling_latency_s":0.686466,"redis_calls":13,"redis_duration_s":0.002673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014567,"completed_at":"2026-09-03T10:18:08.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 306.282803] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"815fb00151ef22f5c0cfceca","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-815fb00151ef22f5c0cfceca: start","job_status":"start","queue_duration_s":0.702363,"scheduling_latency_s":0.702378}1482gitlab # [ 306.295288] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"815fb00151ef22f5c0cfceca","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-815fb00151ef22f5c0cfceca: done: 0.014879 sec","job_status":"done","queue_duration_s":0.702363,"scheduling_latency_s":0.702378,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001819,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014879,"completed_at":"2026-09-03T10:18:08.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 306.335710] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44bdd7797769cda159b0ca79","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44bdd7797769cda159b0ca79: start","job_status":"start","queue_duration_s":0.718645,"scheduling_latency_s":0.71866}1484gitlab # [ 306.348285] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44bdd7797769cda159b0ca79","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44bdd7797769cda159b0ca79: done: 0.014601 sec","job_status":"done","queue_duration_s":0.718645,"scheduling_latency_s":0.71866,"redis_calls":13,"redis_duration_s":0.002632,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014601,"completed_at":"2026-09-03T10:18:08.214Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 306.386995] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8797a0c2bb8f81d06dae4a1e","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8797a0c2bb8f81d06dae4a1e: start","job_status":"start","queue_duration_s":0.734853,"scheduling_latency_s":0.734868}1486gitlab # [ 306.399508] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8797a0c2bb8f81d06dae4a1e","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8797a0c2bb8f81d06dae4a1e: done: 0.016215 sec","job_status":"done","queue_duration_s":0.734853,"scheduling_latency_s":0.734868,"redis_calls":13,"redis_duration_s":0.0026569999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001731,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016215,"completed_at":"2026-09-03T10:18:08.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 306.438938] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a670be89bdfefa6f68fb2d79","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a670be89bdfefa6f68fb2d79: start","job_status":"start","queue_duration_s":0.752461,"scheduling_latency_s":0.752476}1488gitlab # [ 306.451129] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a670be89bdfefa6f68fb2d79","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a670be89bdfefa6f68fb2d79: done: 0.015182 sec","job_status":"done","queue_duration_s":0.752461,"scheduling_latency_s":0.752476,"redis_calls":13,"redis_duration_s":0.0027489999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015182,"completed_at":"2026-09-03T10:18:08.248Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 306.490583] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b2173b11dd7e3a4c7b0ec07","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b2173b11dd7e3a4c7b0ec07: start","job_status":"start","queue_duration_s":0.769003,"scheduling_latency_s":0.769018}1490gitlab # [ 306.503106] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b2173b11dd7e3a4c7b0ec07","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b2173b11dd7e3a4c7b0ec07: done: 0.015041 sec","job_status":"done","queue_duration_s":0.769003,"scheduling_latency_s":0.769018,"redis_calls":13,"redis_duration_s":0.002744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001819,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015041,"completed_at":"2026-09-03T10:18:08.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 306.543235] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1271f213cb9c8594823af0ff","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1271f213cb9c8594823af0ff: start","job_status":"start","queue_duration_s":0.78531,"scheduling_latency_s":0.785324}1492gitlab # [ 306.555671] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1271f213cb9c8594823af0ff","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1271f213cb9c8594823af0ff: done: 0.015347 sec","job_status":"done","queue_duration_s":0.78531,"scheduling_latency_s":0.785324,"redis_calls":13,"redis_duration_s":0.002504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001583,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015347,"completed_at":"2026-09-03T10:18:08.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 306.594785] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af5b64d767d270fbfaa2e88f","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af5b64d767d270fbfaa2e88f: start","job_status":"start","queue_duration_s":0.802035,"scheduling_latency_s":0.80205}1494gitlab # [ 306.606906] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af5b64d767d270fbfaa2e88f","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af5b64d767d270fbfaa2e88f: done: 0.015218 sec","job_status":"done","queue_duration_s":0.802035,"scheduling_latency_s":0.80205,"redis_calls":13,"redis_duration_s":0.002784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015218,"completed_at":"2026-09-03T10:18:08.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 306.645675] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d11ad333e3bbcf0e61705b4","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d11ad333e3bbcf0e61705b4: start","job_status":"start","queue_duration_s":0.818698,"scheduling_latency_s":0.818712}1496gitlab # [ 306.658386] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d11ad333e3bbcf0e61705b4","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d11ad333e3bbcf0e61705b4: done: 0.014876 sec","job_status":"done","queue_duration_s":0.818698,"scheduling_latency_s":0.818712,"redis_calls":13,"redis_duration_s":0.0026449999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001741,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014876,"completed_at":"2026-09-03T10:18:08.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 306.697145] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fcc6ea7527fbbebe401de5b","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fcc6ea7527fbbebe401de5b: start","job_status":"start","queue_duration_s":0.834918,"scheduling_latency_s":0.835191}1498gitlab # [ 306.709206] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fcc6ea7527fbbebe401de5b","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fcc6ea7527fbbebe401de5b: done: 0.014951 sec","job_status":"done","queue_duration_s":0.834918,"scheduling_latency_s":0.835191,"redis_calls":13,"redis_duration_s":0.002606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001716,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014951,"completed_at":"2026-09-03T10:18:08.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 306.749679] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed59bd4d43a982ac9eff379a","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed59bd4d43a982ac9eff379a: start","job_status":"start","queue_duration_s":0.851224,"scheduling_latency_s":0.851239}1500gitlab # [ 306.786939] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed59bd4d43a982ac9eff379a","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed59bd4d43a982ac9eff379a: done: 0.014756 sec","job_status":"done","queue_duration_s":0.851224,"scheduling_latency_s":0.851239,"redis_calls":13,"redis_duration_s":0.002644,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014756,"completed_at":"2026-09-03T10:18:08.346Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 306.874877] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7ca9ac7c54337fedf34c279","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7ca9ac7c54337fedf34c279: start","job_status":"start","queue_duration_s":0.867327,"scheduling_latency_s":0.867341}1502gitlab # [ 306.909739] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7ca9ac7c54337fedf34c279","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7ca9ac7c54337fedf34c279: done: 0.015684 sec","job_status":"done","queue_duration_s":0.867327,"scheduling_latency_s":0.867341,"redis_calls":13,"redis_duration_s":0.0034260000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015684,"completed_at":"2026-09-03T10:18:08.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 306.981074] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14cbb3064e60f9220570ddcb","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14cbb3064e60f9220570ddcb: start","job_status":"start","queue_duration_s":0.884378,"scheduling_latency_s":0.884393}1504gitlab # [ 307.018019] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14cbb3064e60f9220570ddcb","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14cbb3064e60f9220570ddcb: done: 0.014754 sec","job_status":"done","queue_duration_s":0.884378,"scheduling_latency_s":0.884393,"redis_calls":13,"redis_duration_s":0.0027069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014754,"completed_at":"2026-09-03T10:18:08.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 307.085939] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54d466ea5d5464ac54944613","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54d466ea5d5464ac54944613: start","job_status":"start","queue_duration_s":0.900786,"scheduling_latency_s":0.900801}1506gitlab # [ 307.099900] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54d466ea5d5464ac54944613","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54d466ea5d5464ac54944613: done: 0.014757 sec","job_status":"done","queue_duration_s":0.900786,"scheduling_latency_s":0.900801,"redis_calls":13,"redis_duration_s":0.0029249999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001171,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001754,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014757,"completed_at":"2026-09-03T10:18:08.396Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 307.140495] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f398142afe4c0ec25cb0776e","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f398142afe4c0ec25cb0776e: start","job_status":"start","queue_duration_s":0.916878,"scheduling_latency_s":0.916894}1508gitlab # [ 307.152803] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f398142afe4c0ec25cb0776e","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f398142afe4c0ec25cb0776e: done: 0.014777 sec","job_status":"done","queue_duration_s":0.916878,"scheduling_latency_s":0.916894,"redis_calls":13,"redis_duration_s":0.002688,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001743,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014777,"completed_at":"2026-09-03T10:18:08.412Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 307.191135] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba1a34b507ed73f276059bd2","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba1a34b507ed73f276059bd2: start","job_status":"start","queue_duration_s":0.93303,"scheduling_latency_s":0.933045}1510gitlab # [ 307.203540] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba1a34b507ed73f276059bd2","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba1a34b507ed73f276059bd2: done: 0.014982 sec","job_status":"done","queue_duration_s":0.93303,"scheduling_latency_s":0.933045,"redis_calls":13,"redis_duration_s":0.002695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014982,"completed_at":"2026-09-03T10:18:08.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 307.242122] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fc6c319bb673bd8b41fe642","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fc6c319bb673bd8b41fe642: start","job_status":"start","queue_duration_s":0.949362,"scheduling_latency_s":0.949402}1512gitlab # [ 307.254491] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fc6c319bb673bd8b41fe642","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fc6c319bb673bd8b41fe642: done: 0.015285 sec","job_status":"done","queue_duration_s":0.949362,"scheduling_latency_s":0.949402,"redis_calls":13,"redis_duration_s":0.0027029999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015285,"completed_at":"2026-09-03T10:18:08.445Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 307.293434] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef2c61a4e70fcd668f6f68a0","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef2c61a4e70fcd668f6f68a0: start","job_status":"start","queue_duration_s":0.965948,"scheduling_latency_s":0.965962}1514gitlab # [ 307.305827] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef2c61a4e70fcd668f6f68a0","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef2c61a4e70fcd668f6f68a0: done: 0.015138 sec","job_status":"done","queue_duration_s":0.965948,"scheduling_latency_s":0.965962,"redis_calls":13,"redis_duration_s":0.003033,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015138,"completed_at":"2026-09-03T10:18:08.461Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 307.345189] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4b3a2ac59d11e1a2b004585","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4b3a2ac59d11e1a2b004585: start","job_status":"start","queue_duration_s":0.982461,"scheduling_latency_s":0.982476}1516gitlab # [ 307.357429] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4b3a2ac59d11e1a2b004585","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4b3a2ac59d11e1a2b004585: done: 0.014806 sec","job_status":"done","queue_duration_s":0.982461,"scheduling_latency_s":0.982476,"redis_calls":13,"redis_duration_s":0.002637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014806,"completed_at":"2026-09-03T10:18:08.478Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 307.395745] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2585a67406689ad6d0de3762","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2585a67406689ad6d0de3762: start","job_status":"start","queue_duration_s":0.998608,"scheduling_latency_s":0.998623}1518gitlab # [ 307.408069] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2585a67406689ad6d0de3762","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2585a67406689ad6d0de3762: done: 0.014899 sec","job_status":"done","queue_duration_s":0.998608,"scheduling_latency_s":0.998623,"redis_calls":13,"redis_duration_s":0.002623,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014899,"completed_at":"2026-09-03T10:18:08.494Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 307.446860] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68abb059db8f23502f34aeeb","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68abb059db8f23502f34aeeb: start","job_status":"start","queue_duration_s":1.015135,"scheduling_latency_s":1.01515}1520gitlab # [ 307.459384] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68abb059db8f23502f34aeeb","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68abb059db8f23502f34aeeb: done: 0.015848 sec","job_status":"done","queue_duration_s":1.015135,"scheduling_latency_s":1.01515,"redis_calls":13,"redis_duration_s":0.0027,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015848,"completed_at":"2026-09-03T10:18:08.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 307.498198] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9ed8b61b7a79708fd9c3297","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9ed8b61b7a79708fd9c3297: start","job_status":"start","queue_duration_s":1.032403,"scheduling_latency_s":1.032417}1522gitlab # [ 307.510579] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9ed8b61b7a79708fd9c3297","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9ed8b61b7a79708fd9c3297: done: 0.014904 sec","job_status":"done","queue_duration_s":1.032403,"scheduling_latency_s":1.032417,"redis_calls":13,"redis_duration_s":0.002776,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014904,"completed_at":"2026-09-03T10:18:08.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 307.550951] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"816210232e11d0b86aea6ed6","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-816210232e11d0b86aea6ed6: start","job_status":"start","queue_duration_s":1.048636,"scheduling_latency_s":1.048651}1524gitlab # [ 307.563145] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"816210232e11d0b86aea6ed6","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-816210232e11d0b86aea6ed6: done: 0.014693 sec","job_status":"done","queue_duration_s":1.048636,"scheduling_latency_s":1.048651,"redis_calls":13,"redis_duration_s":0.002914,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014693,"completed_at":"2026-09-03T10:18:08.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 307.602348] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f070eae52e9bb7f96a6e200f","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f070eae52e9bb7f96a6e200f: start","job_status":"start","queue_duration_s":1.064704,"scheduling_latency_s":1.064718}1526gitlab # [ 307.614587] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f070eae52e9bb7f96a6e200f","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f070eae52e9bb7f96a6e200f: done: 0.015092 sec","job_status":"done","queue_duration_s":1.064704,"scheduling_latency_s":1.064718,"redis_calls":13,"redis_duration_s":0.002993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015092,"completed_at":"2026-09-03T10:18:08.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 307.653908] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728e21354b29df05c643510c","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728e21354b29df05c643510c: start","job_status":"start","queue_duration_s":1.081406,"scheduling_latency_s":1.08142}1528gitlab # [ 307.665971] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728e21354b29df05c643510c","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728e21354b29df05c643510c: done: 0.014942 sec","job_status":"done","queue_duration_s":1.081406,"scheduling_latency_s":1.08142,"redis_calls":13,"redis_duration_s":0.002521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001601,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014942,"completed_at":"2026-09-03T10:18:08.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 307.705300] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be72f5ac7dd2da82f03f50bd","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be72f5ac7dd2da82f03f50bd: start","job_status":"start","queue_duration_s":1.097775,"scheduling_latency_s":1.097791}1530gitlab # [ 307.717617] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be72f5ac7dd2da82f03f50bd","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be72f5ac7dd2da82f03f50bd: done: 0.015005 sec","job_status":"done","queue_duration_s":1.097775,"scheduling_latency_s":1.097791,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015005,"completed_at":"2026-09-03T10:18:08.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 307.757504] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd1577cdd38223ed9f5eda92","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd1577cdd38223ed9f5eda92: start","job_status":"start","queue_duration_s":1.113919,"scheduling_latency_s":1.113935}1532gitlab # [ 307.769692] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd1577cdd38223ed9f5eda92","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd1577cdd38223ed9f5eda92: done: 0.015096 sec","job_status":"done","queue_duration_s":1.113919,"scheduling_latency_s":1.113935,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015096,"completed_at":"2026-09-03T10:18:08.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 307.808730] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d561f16069856c6e814121ea","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d561f16069856c6e814121ea: start","job_status":"start","queue_duration_s":1.130367,"scheduling_latency_s":1.130383}1534gitlab # [ 307.821106] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d561f16069856c6e814121ea","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d561f16069856c6e814121ea: done: 0.015086 sec","job_status":"done","queue_duration_s":1.130367,"scheduling_latency_s":1.130383,"redis_calls":13,"redis_duration_s":0.001579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015086,"completed_at":"2026-09-03T10:18:08.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 307.860717] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68dde7ffd5947e3e39209c1e","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68dde7ffd5947e3e39209c1e: start","job_status":"start","queue_duration_s":1.146543,"scheduling_latency_s":1.146558}1536gitlab # [ 307.873190] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68dde7ffd5947e3e39209c1e","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68dde7ffd5947e3e39209c1e: done: 0.014144 sec","job_status":"done","queue_duration_s":1.146543,"scheduling_latency_s":1.146558,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014144,"completed_at":"2026-09-03T10:18:08.641Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 307.912497] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e9c63e0bb2edbdb18a39fb","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e9c63e0bb2edbdb18a39fb: start","job_status":"start","queue_duration_s":1.161772,"scheduling_latency_s":1.161787}1538gitlab # [ 307.924479] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e9c63e0bb2edbdb18a39fb","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e9c63e0bb2edbdb18a39fb: done: 0.014978 sec","job_status":"done","queue_duration_s":1.161772,"scheduling_latency_s":1.161787,"redis_calls":13,"redis_duration_s":0.001446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014978,"completed_at":"2026-09-03T10:18:08.657Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 307.964218] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db20f762138f245b304bb35a","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db20f762138f245b304bb35a: start","job_status":"start","queue_duration_s":1.178134,"scheduling_latency_s":1.17815}1540gitlab # [ 307.976253] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db20f762138f245b304bb35a","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db20f762138f245b304bb35a: done: 0.014594 sec","job_status":"done","queue_duration_s":1.178134,"scheduling_latency_s":1.17815,"redis_calls":13,"redis_duration_s":0.001502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014594,"completed_at":"2026-09-03T10:18:08.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 308.014418] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31f5dba181ef17745842dfae","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31f5dba181ef17745842dfae: start","job_status":"start","queue_duration_s":1.193841,"scheduling_latency_s":1.193857}1542gitlab # [ 308.026682] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31f5dba181ef17745842dfae","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31f5dba181ef17745842dfae: done: 0.014098 sec","job_status":"done","queue_duration_s":1.193841,"scheduling_latency_s":1.193857,"redis_calls":13,"redis_duration_s":0.0015049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014098,"completed_at":"2026-09-03T10:18:08.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 308.064873] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf905325e7d7a3cc2ba19df2","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf905325e7d7a3cc2ba19df2: start","job_status":"start","queue_duration_s":1.20901,"scheduling_latency_s":1.209024}1544gitlab # [ 308.077074] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf905325e7d7a3cc2ba19df2","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf905325e7d7a3cc2ba19df2: done: 0.013966 sec","job_status":"done","queue_duration_s":1.20901,"scheduling_latency_s":1.209024,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013966,"completed_at":"2026-09-03T10:18:08.703Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 308.115628] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be2fab7847c5fc365996bb82","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be2fab7847c5fc365996bb82: start","job_status":"start","queue_duration_s":1.22404,"scheduling_latency_s":1.224054}1546gitlab # [ 308.128111] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be2fab7847c5fc365996bb82","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be2fab7847c5fc365996bb82: done: 0.013968 sec","job_status":"done","queue_duration_s":1.22404,"scheduling_latency_s":1.224054,"redis_calls":13,"redis_duration_s":0.0014290000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013968,"completed_at":"2026-09-03T10:18:08.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 308.167696] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab4da49a9bd43ae34cc1a1e3","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab4da49a9bd43ae34cc1a1e3: start","job_status":"start","queue_duration_s":1.239589,"scheduling_latency_s":1.239604}1548gitlab # [ 308.180176] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab4da49a9bd43ae34cc1a1e3","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab4da49a9bd43ae34cc1a1e3: done: 0.020022 sec","job_status":"done","queue_duration_s":1.239589,"scheduling_latency_s":1.239604,"redis_calls":13,"redis_duration_s":0.001514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020022,"completed_at":"2026-09-03T10:18:08.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 308.218435] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1add68f77c586487159b0d1","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1add68f77c586487159b0d1: start","job_status":"start","queue_duration_s":1.260696,"scheduling_latency_s":1.260712}1550gitlab # [ 308.230752] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1add68f77c586487159b0d1","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1add68f77c586487159b0d1: done: 0.01434 sec","job_status":"done","queue_duration_s":1.260696,"scheduling_latency_s":1.260712,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01434,"completed_at":"2026-09-03T10:18:08.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 308.270001] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"676bbfb1b1b49c8f232217c6","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-676bbfb1b1b49c8f232217c6: start","job_status":"start","queue_duration_s":1.276386,"scheduling_latency_s":1.276403}1552gitlab # [ 308.282966] redis[836]: 10 changes in 300 seconds. Saving...1553gitlab # [ 308.283983] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"676bbfb1b1b49c8f232217c6","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-676bbfb1b1b49c8f232217c6: done: 0.014401 sec","job_status":"done","queue_duration_s":1.276386,"scheduling_latency_s":1.276403,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014401,"completed_at":"2026-09-03T10:18:08.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554gitlab # [ 308.322130] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaeac52ccd9e8e31e2584221","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaeac52ccd9e8e31e2584221: start","job_status":"start","queue_duration_s":1.291821,"scheduling_latency_s":1.291837}1555gitlab # [ 308.334391] redis[836]: Background saving started by pid 33971556gitlab # [ 308.335339] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaeac52ccd9e8e31e2584221","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaeac52ccd9e8e31e2584221: done: 0.013906 sec","job_status":"done","queue_duration_s":1.291821,"scheduling_latency_s":1.291837,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013906,"completed_at":"2026-09-03T10:18:08.786Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 308.374754] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6082ce8d36954d1b95c3d003","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6082ce8d36954d1b95c3d003: start","job_status":"start","queue_duration_s":1.306775,"scheduling_latency_s":1.30679}1558gitlab # [ 308.386992] redis[3397]: BGSAVE done, 454 keys saved, 0 keys skipped, 118219 bytes written.1559gitlab # [ 308.388221] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:08.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6082ce8d36954d1b95c3d003","created_at":"2026-09-03T10:18:07.350Z","trace_propagation_headers":{"sentry-trace":"09050036d5b04799a8b3e48c129ed1e2-4f3257900289419a","baggage":"sentry-trace_id=09050036d5b04799a8b3e48c129ed1e2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"461de15beddb457a38d74153bf4a584b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:07.480Z","job_size_bytes":2,"pid":2153,"sidekiq_tid":"m7md","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6082ce8d36954d1b95c3d003: done: 0.014693 sec","job_status":"done","queue_duration_s":1.306775,"scheduling_latency_s":1.30679,"redis_calls":13,"redis_duration_s":0.001442,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014693,"completed_at":"2026-09-03T10:18:08.802Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560gitlab # [ 308.426295] redis[3397]: DB saved on disk1561gitlab # [ 308.426981] redis[3397]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1562gitlab # [ 308.428110] redis[836]: Background saving terminated with success1563gitlab # [ 310.519515] postgres[931]: [931] LOG: checkpoint starting: time1564gitlab # [ 311.699345] gitaly[958]: time="2026-09-03T10:18:14.547Z" level=info msg="burden monitor stats" pid=9581565gitlab # [ 327.474711] postgres[3451]: [3451] ERROR: relation "table_without_models" does not exist1566gitlab # [ 327.475302] postgres[3451]: [3451] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1567gitlab # [ 333.347504] runuser[3113]: pam_unix(runuser:session): session closed for user gitlab1568gitlab # [ 333.352113] setup-gitlab-start[3083]: + systemctl restart gitlab-sidekiq.service1569gitlab # [ 333.389219] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:36.237Z","message":"Shutting down"}1570gitlab # [ 333.394226] systemd[1]: Stopping gitlab-sidekiq.service...1571gitlab # [ 333.396828] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:36.238Z","message":"Scheduler exiting..."}1572gitlab # [ 333.401214] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:36.240Z","message":"Terminating quiet threads for default capsule"}1573gitlab # [ 333.406360] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:36.241Z","message":"Scheduler exiting..."}1574gitlab # [ 333.893544] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:36.743Z","message":"Pausing to allow jobs to finish..."}1575gitlab # [ 336.398080] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:39.246Z","message":"Bye!"}1576gitlab # [ 336.437943] sidekiq-cluster[2153]: {"severity":"INFO","time":"2026-09-03T10:18:39.287Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2153,"worker_id":"sidekiq_0","memwd_rss_bytes":970756096,"retry":0}1577gitlab # [ 336.978164] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1578gitlab # [ 336.981278] systemd[1]: Stopped gitlab-sidekiq.service.1579gitlab # [ 336.983585] systemd[1]: gitlab-sidekiq.service: Consumed 1min 58.354s CPU time over 3min 25.297s wall clock time, 954.5M memory peak, 4.4M read from disk.1580gitlab # [ 336.989734] systemd[1]: Started gitlab-sidekiq.service.1581gitlab # [ 336.992944] setup-gitlab-start[3083]: + mkdir -p /var/lib/secrets1582gitlab # [ 336.995454] setup-gitlab-start[3083]: + echo nixbot-test-token-of-20-chars1583gitlab # [ 336.998359] setup-gitlab-start[3083]: + chmod 644 /var/lib/secrets/gitlab-token1584gitlab # [ 337.001384] setup-gitlab-start[3083]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1585gitlab # [ 338.716239] gitaly[958]: time="2026-09-03T10:18:41.564Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:51.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.563" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586gitlab # [ 338.746129] gitaly[958]: time="2026-09-03T10:18:41.565Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:51.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.563" grpc.time_ms=1.796 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1587gitlab # [ 338.950808] gitaly[958]: time="2026-09-03T10:18:41.800Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:51.249" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.800" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1588gitlab # [ 339.012604] gitaly[958]: time="2026-09-03T10:18:41.801Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:51.249" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.800" grpc.time_ms=0.669 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1589gitlab # [ 339.068860] gitaly[958]: time="2026-09-03T10:18:41.918Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 diskcache=6abc0d91-7c81-4180-8647-d8c55507663a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:18:51.265" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.816" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 339.095351] gitaly[958]: time="2026-09-03T10:18:41.919Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=36 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=32 command.real_time_ms=78 command.spawn_token_fork_ms=5 command.system_time_ms=30 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:18:51.265" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.816" grpc.time_ms=102.486 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1591gitlab # [ 339.119101] gitaly[958]: time="2026-09-03T10:18:41.947Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:51.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.946" pid=958 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1592gitlab # [ 339.133740] gitaly[958]: time="2026-09-03T10:18:41.947Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:51.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.946" grpc.time_ms=1.473 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593gitlab # [ 339.148179] gitaly[958]: time="2026-09-03T10:18:41.974Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:18:51.016" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:41.957" grpc.time_ms=16.738 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594gitlab # [ 339.233632] gitaly[958]: time="2026-09-03T10:18:42.083Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:52.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.082" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595gitlab # [ 339.264692] gitaly[958]: time="2026-09-03T10:18:42.083Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:52.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.082" grpc.time_ms=0.82 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596gitlab # [ 339.294578] gitaly[958]: time="2026-09-03T10:18:42.130Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 diskcache=84eaa7cc-b53e-4c49-bca6-e063e9f1d6e1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:18:52.006" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.087" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1597gitlab # [ 339.315441] gitaly[958]: time="2026-09-03T10:18:42.130Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=32 command.real_time_ms=21 command.spawn_token_fork_ms=6 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-03T10:18:52.006" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.087" grpc.time_ms=42.93 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598gitlab # [ 339.337881] gitaly[958]: time="2026-09-03T10:18:42.141Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:52.059" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.140" pid=958 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1599gitlab # [ 339.355503] gitaly[958]: time="2026-09-03T10:18:42.141Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:52.059" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.140" grpc.time_ms=1.388 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1600gitlab # [ 339.372339] gitaly[958]: time="2026-09-03T10:18:42.159Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:18:52.067" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.148" grpc.time_ms=11.072 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1601gitlab # [ 339.391726] gitaly[958]: time="2026-09-03T10:18:42.206Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:18:52.125" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.205" grpc.time_ms=0.214 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602gitlab # [ 339.405411] gitaly[958]: time="2026-09-03T10:18:42.215Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:18:52.134" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.215" grpc.time_ms=0.417 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 340.142537] gitaly[958]: time="2026-09-03T10:18:42.991Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCE6W7TJY9BAD9P307M9H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:18:52.033" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:42.973" grpc.time_ms=17.227 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604gitlab # [ 340.639987] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:43 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 36091605gitlab # [ 340.647227] setup-gitlab-start[3083]: + mkdir -p /tmp/test-flake1606gitlab # [ 340.649985] setup-gitlab-start[3083]: + cd /tmp/test-flake1607gitlab # [ 340.652490] setup-gitlab-start[3083]: + git init -b master1608gitlab # [ 340.679398] setup-gitlab-start[3570]: Initialized empty Git repository in /tmp/test-flake/.git/1609gitlab # [ 340.683109] setup-gitlab-start[3083]: + git config user.name test1610gitlab # [ 340.686415] setup-gitlab-start[3083]: + git config user.email test@example.com1611gitlab # [ 340.693171] setup-gitlab-start[3083]: + cat1612gitlab # [ 340.698605] setup-gitlab-start[3083]: + git add flake.nix1613gitlab # [ 340.707411] setup-gitlab-start[3083]: + git commit -m 'initial commit'1614gitlab # [ 340.743461] setup-gitlab-start[3575]: [master (root-commit) dc6d2c3] initial commit1615gitlab # [ 340.747916] setup-gitlab-start[3575]: 1 file changed, 10 insertions(+)1616gitlab # [ 340.750903] setup-gitlab-start[3575]: create mode 100644 flake.nix1617gitlab # [ 340.753616] setup-gitlab-start[3083]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1618gitlab # [ 340.759380] setup-gitlab-start[3083]: + git push -u origin master1619gitlab # [ 341.103261] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:43 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 2931620gitlab # [ 341.341911] gitaly[958]: time="2026-09-03T10:18:44.191Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEAVJJG6ZWX3WJ4R9K75R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:44.172" grpc.time_ms=18.305 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1621gitlab # [ 341.365762] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:44 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 2381622gitlab # [ 341.967342] gitaly[958]: time="2026-09-03T10:18:44.817Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:18:54.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:44.798" grpc.time_ms=18.25 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1623gitlab # [ 342.006193] sidekiq-cluster[3514]: {"severity":"INFO","time":"2026-09-03T10:18:44.826Z","message":"Starting cluster with 1 processes"}1624gitlab # [ 342.039782] gitaly[958]: time="2026-09-03T10:18:44.889Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-03T10:19:14.077" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-03T10:18:44.878" grpc.time_ms=11.538 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1625gitlab # [ 342.124597] gitaly[958]: time="2026-09-03T10:18:44.974Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E duration_ms=537ns method=POST pid=958 status=200 url="http://unix/api/v4/internal/allowed"1626gitlab # [ 342.165342] gitaly[958]: time="2026-09-03T10:18:45.015Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E duration_ms=40ns method=POST pid=958 status=200 url="http://unix/api/v4/internal/pre_receive"1627gitlab # [ 342.174321] gitaly[958]: time="2026-09-03T10:18:45.015Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:44.434" grpc.time_ms=580.836 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1628gitlab # [ 342.213389] gitaly[958]: time="2026-09-03T10:18:45.063Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:45.061" grpc.time_ms=1.248 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1629gitlab # [ 342.251712] gitaly[958]: time="2026-09-03T10:18:45.101Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:45.100" grpc.time_ms=0.918 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1630gitlab # [ 342.279579] gitaly[958]: time="2026-09-03T10:18:45.129Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:45.126" grpc.time_ms=2.155 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1631gitlab # [ 342.440796] gitaly[958]: time="2026-09-03T10:18:45.290Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:18:55.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:45.273" grpc.time_ms=16.892 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1632gitlab # [ 342.502322] gitaly[958]: time="2026-09-03T10:18:45.352Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=37 command.spawn_token_fork_ms=5 command.system_time_ms=13 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-03T10:18:55.038" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:18:45.309" grpc.time_ms=42.739 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1633gitlab # [ 342.526859] gitaly[958]: time="2026-09-03T10:18:45.376Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E duration_ms=222ns method=POST pid=958 status=200 url="http://unix/api/v4/internal/post_receive"1634gitlab # [ 342.536185] gitaly[958]: time="2026-09-03T10:18:45.377Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:45.153" grpc.time_ms=223.82 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1635gitlab # [ 342.566563] gitaly[958]: time="2026-09-03T10:18:45.387Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E diskcache=d8f2571a-f377-41ca-a3f5-6faab2cfcded grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:44.313" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1636gitlab # [ 342.587598] systemd[1]: Finished setup-gitlab.service.1637gitlab # [ 342.589419] gitlab-workhorse[893]: time="2026-09-03T10:18:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521638gitlab # [ 342.592526] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:45 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 11731639gitlab # [ 342.595740] setup-gitlab-start[3579]: To http://gitlab/root/test-flake.git1640gitlab # [ 342.597222] setup-gitlab-start[3579]: * [new branch] master -> master1641gitlab # [ 342.598621] setup-gitlab-start[3579]: branch 'master' set up to track 'origin/master'.1642gitlab # [ 342.600210] gitaly[958]: time="2026-09-03T10:18:45.388Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=271 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13835 command.oublock=32 command.real_time_ms=1067 command.spawn_token_fork_ms=2 command.system_time_ms=174 command.user_time_ms=97 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:44.313" grpc.time_ms=1074.012 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1643gitlab # [ 342.618544] systemd[1]: Starting nixbot CI service...1644gitlab: (finished: waiting for unit setup-gitlab.service, in 344.18 seconds)1645gitlab: waiting for unit nixbot.service1646gitlab # [ 343.909075] nixbot[3674]: {"ts": "2026-09-03T10:18:46.757Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1647gitlab # [ 343.935765] nixbot[3674]: {"ts": "2026-09-03T10:18:46.785Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1648gitlab # [ 343.952277] nixbot[3674]: {"ts": "2026-09-03T10:18:46.802Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1649gitlab # [ 343.958854] nixbot[3674]: {"ts": "2026-09-03T10:18:46.806Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1650gitlab # [ 343.965646] nixbot[3674]: {"ts": "2026-09-03T10:18:46.810Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1651gitlab # [ 343.971992] nixbot[3674]: {"ts": "2026-09-03T10:18:46.816Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1652gitlab # [ 343.978423] nixbot[3674]: {"ts": "2026-09-03T10:18:46.820Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1653gitlab # [ 343.985080] nixbot[3674]: {"ts": "2026-09-03T10:18:46.822Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1654gitlab # [ 343.992073] nixbot[3674]: {"ts": "2026-09-03T10:18:46.828Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1655gitlab # [ 343.998480] nixbot[3674]: {"ts": "2026-09-03T10:18:46.830Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1656gitlab # [ 344.005432] nixbot[3674]: {"ts": "2026-09-03T10:18:46.835Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1657gitlab # [ 344.011994] nixbot[3674]: {"ts": "2026-09-03T10:18:46.841Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1658gitlab # [ 344.018794] nixbot[3674]: {"ts": "2026-09-03T10:18:46.847Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1659gitlab # [ 344.025578] nixbot[3674]: {"ts": "2026-09-03T10:18:46.849Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1660gitlab # [ 344.032771] nixbot[3674]: {"ts": "2026-09-03T10:18:46.853Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1661gitlab # [ 344.038837] nixbot[3674]: {"ts": "2026-09-03T10:18:46.855Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1662gitlab # [ 344.044092] nixbot[3674]: {"ts": "2026-09-03T10:18:46.857Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1663gitlab # [ 344.049000] nixbot[3674]: {"ts": "2026-09-03T10:18:46.859Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1664gitlab # [ 344.053666] nixbot[3674]: {"ts": "2026-09-03T10:18:46.863Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1665gitlab # [ 344.058223] nixbot[3674]: {"ts": "2026-09-03T10:18:46.865Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1666gitlab # [ 344.062762] nixbot[3674]: {"ts": "2026-09-03T10:18:46.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1667gitlab # [ 344.067473] nixbot[3674]: {"ts": "2026-09-03T10:18:46.872Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1668gitlab # [ 344.077720] nixbot[3674]: {"ts": "2026-09-03T10:18:46.874Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1669gitlab # [ 344.081608] nixbot[3674]: {"ts": "2026-09-03T10:18:46.875Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1670gitlab # [ 344.085147] nixbot[3674]: {"ts": "2026-09-03T10:18:46.876Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1671gitlab # [ 344.089313] nixbot[3674]: {"ts": "2026-09-03T10:18:46.878Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1672gitlab # [ 344.093635] nixbot[3674]: {"ts": "2026-09-03T10:18:46.880Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1673gitlab # [ 344.097854] nixbot[3674]: {"ts": "2026-09-03T10:18:46.881Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1674gitlab # [ 344.102143] nixbot[3674]: {"ts": "2026-09-03T10:18:46.882Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1675gitlab # [ 344.421031] nixbot[3674]: INFO: Started server process [3674]1676gitlab # [ 344.423758] nixbot[3674]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1677gitlab # [ 344.445824] nixbot[3674]: INFO: ::ffff:127.0.0.1:39002 - "GET /health HTTP/1.1" 200 OK1678gitlab # [ 344.454462] systemd[1]: Started nixbot CI service.1679gitlab # [ 344.455268] systemd[1]: Reached target Multi-User System.1680gitlab # [ 344.456056] systemd[1]: Startup finished in 783ms (kernel) + 3.351s (initrd) + 5min 40.320s (userspace) = 5min 44.455s.1681gitlab: (finished: waiting for unit nixbot.service, in 1.10 seconds)1682gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1683gitlab # [ 344.862042] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:47 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 4161684gitlab # [ 344.882423] nixbot[3674]: {"ts": "2026-09-03T10:18:47.721Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1685gitlab # [ 344.891279] nixbot[3674]: INFO: ::ffff:127.0.0.1:39018 - "GET /health HTTP/1.1" 200 OK1686gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.07 seconds)1687subtest: project discovered and webhook registered1688gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1689gitlab # [ 344.911597] nixbot[3674]: {"ts": "2026-09-03T10:18:47.760Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1690gitlab # [ 345.112030] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 1651691gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.22 seconds)1692gitlab # [ 345.240321] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 3101693gitlab # [ 345.252353] nixbot[3674]: {"ts": "2026-09-03T10:18:48.089Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1694gitlab # [ 345.262299] nixbot[3674]: {"ts": "2026-09-03T10:18:48.100Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1695gitlab # [ 345.583727] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 3201696gitlab # [ 345.596288] nixbot[3674]: {"ts": "2026-09-03T10:18:48.442Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1697gitlab # [ 345.897239] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:48 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2901698gitlab # [ 345.900266] nixbot[3674]: {"ts": "2026-09-03T10:18:48.748Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1699gitlab # [ 346.104089] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 1871700gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1701gitlab # [ 346.111932] nixbot[3674]: {"ts": "2026-09-03T10:18:48.955Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1702gitlab # [ 346.212309] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 961703gitlab # [ 346.218717] nixbot[3674]: {"ts": "2026-09-03T10:18:49.058Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1704gitlab # [ 346.308245] gitaly[958]: time="2026-09-03T10:18:49.156Z" level=info msg="finished unary call with code OK" catfile.duration_ms=7 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=7 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEFV8Q4N3HTKGT9MWZT4H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-03T10:19:19.026" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:18:49.127" grpc.time_ms=28.543 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.24 seconds)1706(finished: subtest: project discovered and webhook registered, in 1.46 seconds)1707subtest: push delivers the webhook1708gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1709gitlab # [ 346.352440] gitaly[958]: time="2026-09-03T10:18:49.184Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEFV8Q4N3HTKGT9MWZT4H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:19:19.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:18:49.184" grpc.time_ms=0.64 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1710gitlab # [ 346.372359] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 1491711gitlab # [ 346.382393] gitaly[958]: time="2026-09-03T10:18:49.232Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=6 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEFV8Q4N3HTKGT9MWZT4H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-03T10:18:59.129" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:18:49.211" grpc.time_ms=19.988 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 346.447850] nixbot[3674]: {"ts": "2026-09-03T10:18:49.293Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1713gitlab # [ 346.452720] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2271714gitlab # [ 346.480966] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 461715gitlab # [ 346.568099] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1151716gitlab # [ 346.572419] nixbot[3674]: {"ts": "2026-09-03T10:18:49.415Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1717gitlab # [ 346.580137] nixbot[3674]: {"ts": "2026-09-03T10:18:49.423Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1718gitlab # [ 346.676121] gitaly[958]: time="2026-09-03T10:18:49.525Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEG3M83PTZBP52D75Y8YG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.512" grpc.time_ms=13.545 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1719gitlab # [ 346.696531] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 1961720gitlab # Enumerating objects: 5, done.1721gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1722gitlab # Delta compression using up to 4 threads1723gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1724gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 91.00 KiB/s, done.1725gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1726gitlab # [ 346.748180] gitaly[958]: time="2026-09-03T10:18:49.596Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEG8ERA7MP2HJ8QV9J108 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.581" grpc.time_ms=15.112 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1727gitlab # [ 346.767604] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1101728gitlab # [ 346.872161] gitaly[958]: time="2026-09-03T10:18:49.722Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGBZ1QYRBKHB03HAFAC5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.706" pid=958 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1729gitlab # [ 346.900409] gitaly[958]: time="2026-09-03T10:18:49.722Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGBZ1QYRBKHB03HAFAC5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.706" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1730gitlab # [ 346.926395] gitlab-workhorse[893]: time="2026-09-03T10:18:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131731gitlab # [ 346.929654] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1231732gitlab # [ 346.932589] gitaly[958]: time="2026-09-03T10:18:49.728Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGBZ1QYRBKHB03HAFAC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.706" grpc.time_ms=15.764 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1733gitlab # [ 346.974314] gitaly[958]: time="2026-09-03T10:18:49.823Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGFZA5XEX299N0MEMP9R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.806" pid=958 remote_ip="2001:db8:1::1" request_sha=1f982f80a6e3509aa4b25fbf8d0c21c2ad6d1ba3 response_bytes=223 span.kind=server system=grpc1734gitlab # [ 346.985996] gitlab-workhorse[893]: time="2026-09-03T10:18:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231735gitlab # [ 346.988040] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 961736gitlab # [ 346.990063] gitaly[958]: time="2026-09-03T10:18:49.823Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGFZA5XEX299N0MEMP9R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.806" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1737gitlab # [ 347.004335] gitaly[958]: time="2026-09-03T10:18:49.824Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGFZA5XEX299N0MEMP9R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.806" grpc.time_ms=16.922 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1738gitlab # [ 347.087869] gitaly[958]: time="2026-09-03T10:18:49.937Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-03T10:19:19.825" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-03T10:18:49.926" grpc.time_ms=11.532 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739gitlab # [ 347.105560] gitaly[958]: time="2026-09-03T10:18:49.955Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGN37FBT1XADET4JZ9XV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.944" grpc.time_ms=11.253 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1740gitlab # [ 347.119733] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 651741gitlab # [ 347.137825] gitaly[958]: time="2026-09-03T10:18:49.987Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1KCEGB0170C7ZY19CZVYXFT duration_ms=176ns method=POST pid=958 status=200 url="http://unix/api/v4/internal/allowed"1742gitlab # [ 347.178372] gitaly[958]: time="2026-09-03T10:18:50.028Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1KCEGB0170C7ZY19CZVYXFT duration_ms=39ns method=POST pid=958 status=200 url="http://unix/api/v4/internal/pre_receive"1743gitlab # [ 347.185402] gitaly[958]: time="2026-09-03T10:18:50.029Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:49.810" grpc.time_ms=218.943 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1744gitlab # [ 347.241507] gitaly[958]: time="2026-09-03T10:18:50.091Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:50.090" grpc.time_ms=0.951 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1745gitlab # [ 347.295479] gitaly[958]: time="2026-09-03T10:18:50.145Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:50.144" grpc.time_ms=1.04 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1746gitlab # [ 347.335950] gitaly[958]: time="2026-09-03T10:18:50.185Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:50.184" grpc.time_ms=0.785 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1747gitlab # [ 347.358837] gitaly[958]: time="2026-09-03T10:18:50.208Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGQ7MK8V6QC6K0R8FBAT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.187" pid=958 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1748gitlab # [ 347.369220] gitaly[958]: time="2026-09-03T10:18:50.209Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=10 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGQ7MK8V6QC6K0R8FBAT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.187" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1749gitlab # [ 347.382811] gitlab-workhorse[893]: time="2026-09-03T10:18:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671750gitlab # [ 347.384709] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 2511751gitlab # [ 347.386581] gitaly[958]: time="2026-09-03T10:18:50.212Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=10 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGQ7MK8V6QC6K0R8FBAT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.187" grpc.time_ms=22.075 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1752gitlab # [ 347.451759] gitaly[958]: time="2026-09-03T10:18:50.301Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1KCEGB0170C7ZY19CZVYXFT duration_ms=74ns method=POST pid=958 status=200 url="http://unix/api/v4/internal/post_receive"1753gitlab # [ 347.456801] gitaly[958]: time="2026-09-03T10:18:50.302Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:50.227" grpc.time_ms=75.176 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1754gitlab # [ 347.474225] gitaly[958]: time="2026-09-03T10:18:50.311Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT diskcache=d66ca7b7-a95a-4e0f-9ffa-c9e03b66be0c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.684" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1755gitlab # To http://gitlab/root/test-flake.git1756gitlab # [ 347.484067] gitlab-workhorse[893]: time="2026-09-03T10:18:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521757gitlab # dc6d2c3..05c5fe8 [ 347.486411] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:50 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7551758gitlab # master -> master[ 347.488597] gitaly[958]: time="2026-09-03T10:18:50.311Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=231 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13457 command.oublock=32 command.real_time_ms=621 command.spawn_token_fork_ms=4 command.system_time_ms=138 command.user_time_ms=93 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:49.684" grpc.time_ms=626.411 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1759gitlab # 1760gitlab # [ 347.511117] gitaly[958]: time="2026-09-03T10:18:50.361Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGZ7FDMGFH6CJBXQY008 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.330" pid=958 remote_ip="2001:db8:1::1" request_sha=33bc0e131b4eec69a5d83b92ee56232deb6e2b71 response_bytes=337 span.kind=server system=grpc1761gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.17 seconds)1762gitlab: must succeed: git -C /tmp/test-flake rev-parse master1763gitlab # [ 347.521110] gitaly[958]: time="2026-09-03T10:18:50.361Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1130 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGZ7FDMGFH6CJBXQY008 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.330" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1764gitlab # [ 347.534757] gitlab-workhorse[893]: time="2026-09-03T10:18:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371765gitlab # [ 347.536629] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 1461766gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1767gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1768gitlab # [ 347.538516] gitaly[958]: time="2026-09-03T10:18:50.361Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1130 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGZ7FDMGFH6CJBXQY008 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.330" grpc.time_ms=31.141 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1769gitlab # [ 347.637659] nixbot[3674]: INFO: ::ffff:127.0.0.1:39028 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1770gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.10 seconds)1771gitlab # [ 347.652725] gitaly[958]: time="2026-09-03T10:18:50.502Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEH5VS7HYSJ2RVFKGZZ92 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.493" grpc.time_ms=8.625 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1772gitlab # [ 347.665833] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 741773gitlab # [ 347.874829] gitaly[958]: time="2026-09-03T10:18:50.724Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEH8A4F6TBK42R5VF5PER grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.676" pid=958 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1774gitlab # [ 347.897426] gitaly[958]: time="2026-09-03T10:18:50.725Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=41 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=43 command.spawn_token_fork_ms=2 command.system_time_ms=32 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEH8A4F6TBK42R5VF5PER grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.676" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1775gitlab # [ 347.922569] gitlab-workhorse[893]: time="2026-09-03T10:18:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091776gitlab # [ 347.925669] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 2211777gitlab # [ 347.928701] gitaly[958]: time="2026-09-03T10:18:50.726Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=41 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=43 command.spawn_token_fork_ms=2 command.system_time_ms=32 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEH8A4F6TBK42R5VF5PER grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:18:50.676" grpc.time_ms=49.434 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1778gitlab # [ 348.148127] gitaly[958]: time="2026-09-03T10:18:50.997Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:19:20.195" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:18:50.996" grpc.time_ms=1.403 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779gitlab # [ 348.265655] gitaly[958]: time="2026-09-03T10:18:51.115Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-03T10:19:21.095" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:18:51.096" grpc.time_ms=18.997 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 348.322254] gitaly[958]: time="2026-09-03T10:18:51.172Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:19:01.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:18:51.154" grpc.time_ms=16.836 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1781gitlab # [ 348.397550] gitaly[958]: time="2026-09-03T10:18:51.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:19:01.076" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:18:51.227" grpc.time_ms=18.997 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1782gitlab # [ 348.580867] gitaly[958]: time="2026-09-03T10:18:51.430Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:19:01.159" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:51.429" grpc.time_ms=0.719 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1784gitlab # [ 348.646511] gitaly[958]: time="2026-09-03T10:18:51.496Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:19:01.206" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:18:51.477" grpc.time_ms=18.772 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1785gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1786gitlab # [ 348.701172] nixbot[3674]: INFO: ::ffff:127.0.0.1:39040 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1787gitlab # [ 348.728497] gitaly[958]: time="2026-09-03T10:18:51.578Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:51.577" grpc.time_ms=1.054 pid=958 span.kind=server system=grpc1788gitlab # [ 348.775092] gitaly[958]: time="2026-09-03T10:18:51.625Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:18:51.623" grpc.time_ms=0.988 pid=958 span.kind=server system=grpc1789gitlab # [ 348.809774] gitaly[958]: time="2026-09-03T10:18:51.659Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 diskcache=e32031d6-04db-469b-b11a-7a83bd1bf1d7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:19:01.230" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:51.501" pid=958 ref=refs/keep-around/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc remote_ip="2001:db8:1::1" revision=dc6d2c375b1a031f8f5579a9741b23a7183f1fcc span.kind=server system=grpc username=root1790gitlab # [ 348.834445] gitaly[958]: time="2026-09-03T10:18:51.660Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=119 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5145 command.oublock=8 command.real_time_ms=149 command.spawn_token_fork_ms=16 command.system_time_ms=62 command.user_time_ms=57 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:19:01.230" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:18:51.501" grpc.time_ms=158.216 pid=958 ref=refs/keep-around/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc remote_ip="2001:db8:1::1" revision=dc6d2c375b1a031f8f5579a9741b23a7183f1fcc span.kind=server system=grpc username=root1791gitlab # [ 348.859265] gitaly[958]: time="2026-09-03T10:18:51.669Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:19:21.666" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:18:51.667" grpc.time_ms=1.333 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792gitlab # [ 349.119550] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:18:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 11581793gitlab # [ 349.127805] nixbot[3674]: {"ts": "2026-09-03T10:18:51.970Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc \"HTTP/1.1 201 Created\""}1794gitlab # [ 349.136660] nixbot[3674]: {"ts": "2026-09-03T10:18:51.985Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-dc6d2c375b1a-0"}1795gitlab # [ 349.449850] nixbot[3674]: {"ts": "2026-09-03T10:18:52.299Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-dc6d2c375b1a-0", "argv0": "bwrap"}1796gitlab # [ 349.642066] systemd[1]: Started Nix Daemon.1797gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1798gitlab # [ 349.766821] nixbot[3674]: INFO: ::ffff:127.0.0.1:39052 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1799gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.07 seconds)1800gitlab # [ 349.775282] nix-daemon[3902]: accepted connection from pid 3897, user nixbot1801gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1802gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1803gitlab # [ 350.825056] nixbot[3674]: INFO: ::ffff:127.0.0.1:39060 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1804gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1805gitlab # [ 351.879649] nixbot[3674]: INFO: ::ffff:127.0.0.1:54078 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1806gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1807gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1808gitlab # [ 352.936371] nixbot[3674]: INFO: ::ffff:127.0.0.1:54092 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1809gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1810gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1811gitlab # [ 353.994789] nixbot[3674]: INFO: ::ffff:127.0.0.1:54098 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1812gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1813gitlab # [ 354.224493] nix-daemon[3902]: accepted connection from pid 3929, user nixbot1814gitlab # [ 354.245948] nix-daemon[3902]: accepted connection from pid 3897, user nixbot1815gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1816gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1817gitlab # [ 355.052347] nixbot[3674]: INFO: ::ffff:127.0.0.1:54100 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1818gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1819gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1820gitlab # [ 356.106883] nixbot[3674]: INFO: ::ffff:127.0.0.1:54108 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1821gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1822gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1823gitlab # [ 357.161275] nixbot[3674]: INFO: ::ffff:127.0.0.1:54112 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1824gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1825gitlab # [ 358.215812] nixbot[3674]: INFO: ::ffff:127.0.0.1:54114 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1826gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1827gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1828gitlab # [ 359.271405] nixbot[3674]: INFO: ::ffff:127.0.0.1:54120 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1829gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1830gitlab # [ 359.303468] nix-daemon[3902]: accepted connection from pid 3897, user nixbot1831gitlab # [ 359.517256] gitaly[958]: time="2026-09-03T10:19:02.366Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEWQ0B1JKB2QZYQWZ1QKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:19:32.355" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:19:02.356" grpc.time_ms=9.76 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832gitlab # [ 359.596140] gitaly[958]: time="2026-09-03T10:19:02.445Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEWQ0B1JKB2QZYQWZ1QKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:19:12.156" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:19:02.427" grpc.time_ms=18.047 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833gitlab # [ 359.704880] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:19:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 3121834gitlab # [ 359.712957] nixbot[3674]: {"ts": "2026-09-03T10:19:02.555Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc \"HTTP/1.1 201 Created\""}1835gitlab # [ 359.789080] gitaly[958]: time="2026-09-03T10:19:02.638Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEX11G126QW6P3ZRSMKNZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:19:32.636" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:19:02.637" grpc.time_ms=1.189 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836gitlab # [ 359.896988] gitaly[958]: time="2026-09-03T10:19:02.746Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEX11G126QW6P3ZRSMKNZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:19:12.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:19:02.729" grpc.time_ms=16.968 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837gitlab # [ 359.987000] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:19:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2741838gitlab # [ 360.001076] nixbot[3674]: {"ts": "2026-09-03T10:19:02.846Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc \"HTTP/1.1 201 Created\""}1839gitlab # [ 360.096698] gitaly[958]: time="2026-09-03T10:19:02.946Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEXAJM58FKA1R260A3XA4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:19:32.144" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:19:02.945" grpc.time_ms=1.109 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 360.153314] gitaly[958]: time="2026-09-03T10:19:03.003Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEXAJM58FKA1R260A3XA4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:19:12.264" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:19:02.985" grpc.time_ms=17.669 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841gitlab # [ 360.231710] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:19:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2141842gitlab # [ 360.236338] nixbot[3674]: {"ts": "2026-09-03T10:19:03.083Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc \"HTTP/1.1 201 Created\""}1843gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1844gitlab # [ 360.319531] nixbot[3674]: INFO: ::ffff:127.0.0.1:54136 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1845gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1846gitlab # [ 360.558036] gitaly[958]: time="2026-09-03T10:19:03.407Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEXR5029VN0CW5AZ8D2T4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:19:03.389" grpc.time_ms=17.86 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1847gitlab # [ 360.585514] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:19:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1071848gitlab # [ 360.721464] gitaly[958]: time="2026-09-03T10:19:03.571Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEXWD6APG3V25EKA6HSJ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:19:03.552" pid=958 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1849gitlab # [ 360.752782] gitaly[958]: time="2026-09-03T10:19:03.571Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEXWD6APG3V25EKA6HSJ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:19:03.552" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1850gitlab # [ 360.791686] gitlab-workhorse[893]: time="2026-09-03T10:19:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671851gitlab # [ 360.796297] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:19:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1351852gitlab # [ 360.800350] gitaly[958]: time="2026-09-03T10:19:03.572Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEXWD6APG3V25EKA6HSJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:19:03.552" grpc.time_ms=19.87 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc1853gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1854gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1855gitlab # [ 361.382713] nixbot[3674]: INFO: ::ffff:127.0.0.1:34722 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1856gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1857gitlab # [ 362.439749] nixbot[3674]: INFO: ::ffff:127.0.0.1:34728 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1858gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1859gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1860gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1861gitlab # [ 363.495733] nixbot[3674]: INFO: ::ffff:127.0.0.1:34738 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1862gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1863gitlab # [ 364.549750] nixbot[3674]: INFO: ::ffff:127.0.0.1:34742 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1864gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1865gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1866gitlab # [ 365.605877] nixbot[3674]: INFO: ::ffff:127.0.0.1:34754 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1867gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1868gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1869gitlab # [ 366.663870] nixbot[3674]: INFO: ::ffff:127.0.0.1:34762 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1870gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1871gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1872gitlab # [ 367.720442] nixbot[3674]: INFO: ::ffff:127.0.0.1:34766 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1873gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1874gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1875gitlab # [ 368.777273] nixbot[3674]: INFO: ::ffff:127.0.0.1:34770 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1876gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1877gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1878gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1879gitlab # [ 369.833406] nixbot[3674]: INFO: ::ffff:127.0.0.1:34784 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1880gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1881gitlab # [ 370.886690] nixbot[3674]: INFO: ::ffff:127.0.0.1:43152 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1882gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1883gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1884gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1885gitlab # [ 371.942733] nixbot[3674]: INFO: ::ffff:127.0.0.1:43160 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1886gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1887gitlab # [ 372.995741] nixbot[3674]: INFO: ::ffff:127.0.0.1:43168 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1888gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1889gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1890gitlab # [ 374.051190] nixbot[3674]: INFO: ::ffff:127.0.0.1:43174 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1891gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1892gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1893gitlab # [ 375.106596] nixbot[3674]: INFO: ::ffff:127.0.0.1:43184 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1894gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1895gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1896gitlab # [ 376.162551] nixbot[3674]: INFO: ::ffff:127.0.0.1:43200 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1897gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1898gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1899gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1900gitlab # [ 377.217831] nixbot[3674]: INFO: ::ffff:127.0.0.1:43204 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1901gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1902gitlab # [ 378.271960] nixbot[3674]: INFO: ::ffff:127.0.0.1:43206 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1903gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1904gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1905gitlab # [ 379.309462] nixbot[3674]: INFO: ::ffff:127.0.0.1:43208 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1906gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.04 seconds)1907gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1908gitlab # [ 380.365027] nixbot[3674]: INFO: ::ffff:127.0.0.1:43224 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1909gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1910gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1911gitlab # [ 381.420097] nixbot[3674]: INFO: ::ffff:127.0.0.1:35724 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1912gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1913gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1914gitlab # [ 382.476483] nixbot[3674]: INFO: ::ffff:127.0.0.1:35726 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1915gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1916gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1917gitlab # [ 383.606540] nixbot[3674]: INFO: ::ffff:127.0.0.1:35740 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1918gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.13 seconds)1919gitlab # [ 384.434695] sidekiq-cluster[3622]: 2026-09-03T10:19:27.280Z pid=3622 tid=y32 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3622, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425242624}1920gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1921gitlab # [ 384.750830] nixbot[3674]: INFO: ::ffff:127.0.0.1:35756 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1922gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.14 seconds)1923gitlab # [ 385.074956] sidekiq-cluster[3622]: W, [2026-09-03T10:19:27.924637 #3622] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1924gitlab # [ 385.405064] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:19:28.254Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1925gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1926gitlab # [ 385.900824] nixbot[3674]: INFO: ::ffff:127.0.0.1:35770 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1927gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.15 seconds)1928gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1929gitlab # [ 386.995454] nixbot[3674]: INFO: ::ffff:127.0.0.1:35780 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1930gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.09 seconds)1931gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1932gitlab # [ 388.077890] nixbot[3674]: INFO: ::ffff:127.0.0.1:35788 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1933gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.08 seconds)1934gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1935gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1936gitlab # [ 389.135366] nixbot[3674]: INFO: ::ffff:127.0.0.1:35794 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1937gitlab # [ 389.996382] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:19:32.845Z","message":"GitLab reliable fetch activated!"}1938gitlab # [ 389.996881] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:19:32.845Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1939gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1940gitlab # [ 390.187868] nixbot[3674]: INFO: ::ffff:127.0.0.1:35810 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1941gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1942gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1943gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1944gitlab # [ 391.246802] nixbot[3674]: INFO: ::ffff:127.0.0.1:36138 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1945gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1946gitlab # [ 392.302275] nixbot[3674]: INFO: ::ffff:127.0.0.1:36142 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1947gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1948gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1949gitlab # [ 393.358040] nixbot[3674]: INFO: ::ffff:127.0.0.1:36152 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1950gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1951gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1952gitlab # [ 394.413379] nixbot[3674]: INFO: ::ffff:127.0.0.1:36166 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1953gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1954gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1955gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1956gitlab # [ 395.469230] nixbot[3674]: INFO: ::ffff:127.0.0.1:36172 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1957gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1958gitlab # [ 396.522717] nixbot[3674]: INFO: ::ffff:127.0.0.1:36182 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1959gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1960gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1961gitlab # [ 397.578195] nixbot[3674]: INFO: ::ffff:127.0.0.1:36192 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1962gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1963gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1964gitlab # [ 398.633291] nixbot[3674]: INFO: ::ffff:127.0.0.1:36204 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1965gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1966gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1967gitlab # [ 399.690588] nixbot[3674]: INFO: ::ffff:127.0.0.1:36214 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1968gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1969gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1970gitlab # [ 400.747893] nixbot[3674]: INFO: ::ffff:127.0.0.1:36230 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1971gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1972gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1973gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1974gitlab # [ 401.805996] nixbot[3674]: INFO: ::ffff:127.0.0.1:34804 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1975gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1976gitlab # [ 402.859368] nixbot[3674]: INFO: ::ffff:127.0.0.1:34816 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1977gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1978gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1979gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1980gitlab # [ 403.917376] nixbot[3674]: INFO: ::ffff:127.0.0.1:34824 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1981gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1982gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1983gitlab # [ 404.970660] nixbot[3674]: INFO: ::ffff:127.0.0.1:34830 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1984gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1985gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1986gitlab # [ 406.023304] nixbot[3674]: INFO: ::ffff:127.0.0.1:34842 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1987gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1988gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)1989gitlab # [ 407.075918] nixbot[3674]: INFO: ::ffff:127.0.0.1:34856 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1990gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1991gitlab # [ 408.129194] nixbot[3674]: INFO: ::ffff:127.0.0.1:34872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1992gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1993gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1994gitlab # [ 409.184101] nixbot[3674]: INFO: ::ffff:127.0.0.1:34882 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1995gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1996gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'1997gitlab # [ 410.242269] nixbot[3674]: INFO: ::ffff:127.0.0.1:34898 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK1998gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)1999gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2000gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2001gitlab # [ 411.297506] nixbot[3674]: INFO: ::ffff:127.0.0.1:49914 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2002gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2003gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2004gitlab # [ 412.351782] nixbot[3674]: INFO: ::ffff:127.0.0.1:49916 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2005gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2006gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2007gitlab # [ 413.405976] nixbot[3674]: INFO: ::ffff:127.0.0.1:49928 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2008gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2009gitlab # [ 414.461675] nixbot[3674]: INFO: ::ffff:127.0.0.1:49944 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2010gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2011gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2012gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2013gitlab # [ 415.516905] nixbot[3674]: INFO: ::ffff:127.0.0.1:49946 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2014gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2015gitlab # [ 416.574550] nixbot[3674]: INFO: ::ffff:127.0.0.1:49960 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2016gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2017gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2018gitlab # [ 417.630859] nixbot[3674]: INFO: ::ffff:127.0.0.1:49972 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2019gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2020gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2021gitlab # [ 418.686318] nixbot[3674]: INFO: ::ffff:127.0.0.1:49978 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2022gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2023gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2024gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2025gitlab # [ 419.743497] nixbot[3674]: INFO: ::ffff:127.0.0.1:49994 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2026gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2027gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2028gitlab # [ 420.797281] nixbot[3674]: INFO: ::ffff:127.0.0.1:49996 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2029gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2030gitlab # [ 421.852074] nixbot[3674]: INFO: ::ffff:127.0.0.1:49932 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2031gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2032gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2033gitlab # [ 422.906728] nixbot[3674]: INFO: ::ffff:127.0.0.1:49934 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2034gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2035gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2036gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2037gitlab # [ 423.966155] nixbot[3674]: INFO: ::ffff:127.0.0.1:49946 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2038gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2039gitlab # [ 425.019138] nixbot[3674]: INFO: ::ffff:127.0.0.1:49948 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2040gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2041gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2042gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2043gitlab # [ 426.074204] nixbot[3674]: INFO: ::ffff:127.0.0.1:49956 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2044gitlab # [ 426.126127] postgres[4212]: [4212] ERROR: relation "table_without_models" does not exist2045gitlab # [ 426.129692] postgres[4212]: [4212] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')2046gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2047gitlab # [ 427.128451] nixbot[3674]: INFO: ::ffff:127.0.0.1:49966 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2048gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2049gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2050gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2051gitlab # [ 428.183619] nixbot[3674]: INFO: ::ffff:127.0.0.1:49972 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2052gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2053gitlab # [ 429.235937] nixbot[3674]: INFO: ::ffff:127.0.0.1:49984 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2054gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2055gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2056gitlab # [ 430.291113] nixbot[3674]: INFO: ::ffff:127.0.0.1:49988 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2057gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2058gitlab # [ 430.934504] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:13.783Z","message":"Booted Rails 7.2.3.1 application in production environment"}2059gitlab # [ 430.936611] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:13.786Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}2060gitlab # [ 430.938744] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:13.788Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}2061gitlab # [ 430.940507] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:13.790Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}2062gitlab # [ 430.943879] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:13.793Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}2063gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2064gitlab # [ 431.350205] nixbot[3674]: INFO: ::ffff:127.0.0.1:45908 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2065gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2066gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2067gitlab # [ 432.431467] nixbot[3674]: INFO: ::ffff:127.0.0.1:45922 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2068gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.08 seconds)2069gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2070gitlab # [ 433.488763] nixbot[3674]: INFO: ::ffff:127.0.0.1:45928 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2071gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2072gitlab # [ 433.618411] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:16.466Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5e006fe8f787041c3446743a","created_at":"2026-09-03T10:18:41.997Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.050Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5e006fe8f787041c3446743a: start","job_status":"start","queue_duration_s":94.415841,"scheduling_latency_s":94.415941}2073gitlab # [ 433.877074] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:16.726Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5e006fe8f787041c3446743a","created_at":"2026-09-03T10:18:41.997Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.050Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5e006fe8f787041c3446743a: done: 0.260286 sec","job_status":"done","queue_duration_s":94.415841,"scheduling_latency_s":94.415941,"redis_calls":23,"redis_duration_s":0.007809999999999999,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003999,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002165,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001646,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14818742,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.260286,"completed_at":"2026-09-03T10:20:16.726Z","load_balancing_strategy":"primary","db_duration_s":0.098343,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2074gitlab # [ 433.985458] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:16.728Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a8ac633e1b2521b474de42e6","created_at":"2026-09-03T10:18:42.068Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.075Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a8ac633e1b2521b474de42e6: start","job_status":"start","queue_duration_s":94.653238,"scheduling_latency_s":94.653262}2075gitlab # [ 434.039280] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:16.888Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a8ac633e1b2521b474de42e6","created_at":"2026-09-03T10:18:42.068Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.075Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a8ac633e1b2521b474de42e6: done: 0.160105 sec","job_status":"done","queue_duration_s":94.653238,"scheduling_latency_s":94.653262,"redis_calls":14,"redis_duration_s":0.008006,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00405,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002353,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001603,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12247728,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.160105,"completed_at":"2026-09-03T10:20:16.888Z","load_balancing_strategy":"primary","db_duration_s":0.037169,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2076gitlab # [ 434.116063] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:16.890Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"4b161df6965e27f63c7c45f8","created_at":"2026-09-03T10:18:42.788Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.800Z","job_size_bytes":5,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-4b161df6965e27f63c7c45f8: start","job_status":"start","queue_duration_s":94.09081,"scheduling_latency_s":94.090833}2077gitlab # [ 434.251047] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:17.100Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"4b161df6965e27f63c7c45f8","created_at":"2026-09-03T10:18:42.788Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.800Z","job_size_bytes":5,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-4b161df6965e27f63c7c45f8: done: 0.209702 sec","job_status":"done","queue_duration_s":94.09081,"scheduling_latency_s":94.090833,"redis_calls":12,"redis_duration_s":0.003626,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000859,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00216,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.161687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003908,"duration_s":0.209702,"completed_at":"2026-09-03T10:20:17.100Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041284,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2078gitlab # [ 434.347374] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:17.102Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"9764916ea69e3cff83572152","created_at":"2026-09-03T10:18:42.844Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.851Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-9764916ea69e3cff83572152: start","job_status":"start","queue_duration_s":94.250455,"scheduling_latency_s":94.250473}2079gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2080gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.05 seconds)2081gitlab # [ 434.536100] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:17.384Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"9764916ea69e3cff83572152","created_at":"2026-09-03T10:18:42.844Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.851Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-9764916ea69e3cff83572152: done: 0.28233 sec","job_status":"done","queue_duration_s":94.250455,"scheduling_latency_s":94.250473,"redis_calls":11,"redis_duration_s":0.004154000000000001,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000768,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000849,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.23,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.23,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.174015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00098806,"duration_s":0.28233,"completed_at":"2026-09-03T10:20:17.384Z","load_balancing_strategy":"primary","db_duration_s":0.107569,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082gitlab # [ 434.592366] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:17.389Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"7e9b4949ebaf618c2a4837b5","created_at":"2026-09-03T10:18:42.857Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.863Z","job_size_bytes":89,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-7e9b4949ebaf618c2a4837b5: start","job_status":"start","queue_duration_s":94.525864,"scheduling_latency_s":94.525905}2083gitlab # [ 434.611260] nixbot[3674]: INFO: ::ffff:127.0.0.1:45938 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2084gitlab # [ 434.623053] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:17.471Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"7e9b4949ebaf618c2a4837b5","created_at":"2026-09-03T10:18:42.857Z","trace_propagation_headers":{"sentry-trace":"92b06264f30b4b4e9bdc9691b36b1699-bc4e84becbb54bed","baggage":"sentry-trace_id=92b06264f30b4b4e9bdc9691b36b1699,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCE6W7TJY9BAD9P307M9H0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:42.863Z","job_size_bytes":89,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-7e9b4949ebaf618c2a4837b5: done: 0.08208 sec","job_status":"done","queue_duration_s":94.525864,"scheduling_latency_s":94.525905,"redis_calls":11,"redis_duration_s":0.004561,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001206,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002517,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00084764,"duration_s":0.08208,"completed_at":"2026-09-03T10:20:17.471Z","load_balancing_strategy":"primary","db_duration_s":0.009762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 434.670813] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:17.474Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 dc6d2c375b1a031f8f5579a9741b23a7183f1fcc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"46a43d85b9c7aa59c5ab4259","created_at":"2026-09-03T10:18:45.202Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-29dc082280984205","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:45.234Z","job_size_bytes":185,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-46a43d85b9c7aa59c5ab4259: start","job_status":"start","queue_duration_s":92.239876,"scheduling_latency_s":92.239907}2086gitlab # [ 434.687402] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:17.501Z","message":"changes: \"0000000000000000000000000000000000000000 dc6d2c375b1a031f8f5579a9741b23a7183f1fcc refs/heads/master\\n\""}2087gitlab # [ 434.707534] gitaly[958]: time="2026-09-03T10:20:17.557Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:27.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:17.544" grpc.time_ms=12.297 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088gitlab # [ 434.732444] gitaly[958]: time="2026-09-03T10:20:17.582Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:20:27.024" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:17.565" grpc.time_ms=16.68 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2089gitlab # [ 434.832840] gitaly[958]: time="2026-09-03T10:20:17.682Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-03T10:20:27.122" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:17.662" grpc.time_ms=19.961 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2090gitlab # [ 434.907991] gitaly[958]: time="2026-09-03T10:20:17.757Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:17.756" grpc.time_ms=1.043 pid=958 span.kind=server system=grpc2091gitlab # [ 434.967523] gitaly[958]: time="2026-09-03T10:20:17.817Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:17.816" grpc.time_ms=0.831 pid=958 span.kind=server system=grpc2092gitlab # [ 434.993241] gitaly[958]: time="2026-09-03T10:20:17.830Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E diskcache=a9c2fc75-d5c2-48a8-859b-d0acc040eee8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:20:27.163" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:17.703" pid=958 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2093gitlab # [ 435.015542] gitaly[958]: time="2026-09-03T10:20:17.830Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=137 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5244 command.oublock=8 command.real_time_ms=119 command.spawn_token_fork_ms=1 command.system_time_ms=74 command.user_time_ms=63 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:20:27.163" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:17.703" grpc.time_ms=126.844 pid=958 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root2094gitlab # [ 435.038779] gitaly[958]: time="2026-09-03T10:20:17.857Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-03T10:20:27.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:17.857" grpc.time_ms=0.299 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2095gitlab # [ 435.054039] gitaly[958]: time="2026-09-03T10:20:17.888Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=15 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-03T10:20:27.013" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:17.863" grpc.time_ms=24.736 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096gitlab # [ 435.208371] gitaly[958]: time="2026-09-03T10:20:18.058Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-03T10:20:48.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:18.045" grpc.time_ms=12.195 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097gitlab # [ 435.265828] gitaly[958]: time="2026-09-03T10:20:18.115Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:20:28.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:18.114" grpc.time_ms=0.538 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098gitlab # [ 435.297307] gitaly[958]: time="2026-09-03T10:20:18.131Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-03T10:20:28.009" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-03T10:20:18.119" grpc.time_ms=11.735 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2099gitlab # [ 435.330175] gitaly[958]: time="2026-09-03T10:20:18.176Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:28.048" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:18.158" grpc.time_ms=16.997 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab # [ 435.353572] gitaly[958]: time="2026-09-03T10:20:18.203Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:28.076" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:18.186" grpc.time_ms=16.824 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2101gitlab # [ 435.404153] gitaly[958]: time="2026-09-03T10:20:18.254Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=20 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-03T10:20:48.226" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:18.226" grpc.time_ms=27.419 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2102gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2103gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.07 seconds)2104gitlab # [ 435.606616] nixbot[3674]: INFO: ::ffff:127.0.0.1:45946 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2105gitlab # [ 436.053085] gitaly[958]: time="2026-09-03T10:20:18.901Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-03T10:20:28.048" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:18.899" grpc.time_ms=2.62 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2106gitlab # [ 436.126601] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:18.974Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 dc6d2c375b1a031f8f5579a9741b23a7183f1fcc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"46a43d85b9c7aa59c5ab4259","created_at":"2026-09-03T10:18:45.202Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-29dc082280984205","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:45.234Z","job_size_bytes":185,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-46a43d85b9c7aa59c5ab4259: done: 1.499451 sec","job_status":"done","queue_duration_s":92.239876,"scheduling_latency_s":92.239907,"gitaly_calls":13,"gitaly_duration_s":0.348523,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.067403,"redis_read_bytes":2103,"redis_write_bytes":21747,"redis_cache_calls":15,"redis_cache_duration_s":0.006719,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.009411,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002422,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7835,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006133,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.027027,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.015691,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.29,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.338,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.999554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00098608,"exclusive_lock_hold_duration_s":0.039366479999999995,"duration_s":1.499451,"completed_at":"2026-09-03T10:20:18.973Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.155271,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2107gitlab # [ 436.225648] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:18.977Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","dc6d2c375b1a031f8f5579a9741b23a7183f1fcc 05c5fe8317a89e596d33e6555010cf0e802e5771 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1645bc396b025898c52540d1","created_at":"2026-09-03T10:18:50.275Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-4545d9d9b88c4dcd","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:50.279Z","job_size_bytes":185,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1645bc396b025898c52540d1: start","job_status":"start","queue_duration_s":88.697929,"scheduling_latency_s":88.69797}2108gitlab # [ 436.244154] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.000Z","message":"changes: \"dc6d2c375b1a031f8f5579a9741b23a7183f1fcc 05c5fe8317a89e596d33e6555010cf0e802e5771 refs/heads/master\\n\""}2109gitlab # [ 436.247569] gitaly[958]: time="2026-09-03T10:20:19.048Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:29.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:19.029" grpc.time_ms=18.557 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2110gitlab # [ 436.260858] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.084Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1b5961f57c2c8df69952c17b","created_at":"2026-09-03T10:20:19.081Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-edcf655f045145d3","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"687ad358637c7a52dcd35d60","job_size_bytes":46,"pid":3622,"job_status":"deduplicated","message":"ProjectCacheWorker JID-1b5961f57c2c8df69952c17b: deduplicated: until executing","deduplication.type":"until executing"}2111gitlab # [ 436.275898] gitaly[958]: time="2026-09-03T10:20:19.076Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-03T10:20:29.038" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:19.059" grpc.time_ms=17.643 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab # [ 436.289657] gitaly[958]: time="2026-09-03T10:20:19.130Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-03T10:20:29.001" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:19.112" grpc.time_ms=18.204 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113gitlab # [ 436.309403] gitaly[958]: time="2026-09-03T10:20:19.159Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-03T10:20:49.139" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:19.139" grpc.time_ms=19.315 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2114gitlab # [ 436.347337] gitaly[958]: time="2026-09-03T10:20:19.190Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-03T10:20:29.070" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-03T10:20:19.180" grpc.time_ms=10.196 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2115gitlab # [ 436.372198] gitaly[958]: time="2026-09-03T10:20:19.222Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:29.098" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:19.208" grpc.time_ms=13.176 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 436.547392] gitaly[958]: time="2026-09-03T10:20:19.397Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-03T10:20:29.284" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:19.394" grpc.time_ms=2.501 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2117gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2118gitlab # [ 436.594313] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.442Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","dc6d2c375b1a031f8f5579a9741b23a7183f1fcc 05c5fe8317a89e596d33e6555010cf0e802e5771 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"1645bc396b025898c52540d1","created_at":"2026-09-03T10:18:50.275Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-4545d9d9b88c4dcd","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:18:50.279Z","job_size_bytes":185,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-1645bc396b025898c52540d1: done: 0.465115 sec","job_status":"done","queue_duration_s":88.697929,"scheduling_latency_s":88.69797,"gitaly_calls":7,"gitaly_duration_s":0.129602,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.035443,"redis_read_bytes":500,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.003688,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.00183,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004735,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.011875,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.013315,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.045,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.045,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.290649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001256,"exclusive_lock_hold_duration_s":0.0345203,"duration_s":0.465115,"completed_at":"2026-09-03T10:20:19.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044982,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2119gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2120gitlab # [ 436.677924] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.446Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8c6d251164135c862f970f91","created_at":"2026-09-03T10:18:51.686Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-1a88e0644a674ae4","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:51.723Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8c6d251164135c862f970f91: start","job_status":"start","queue_duration_s":87.722883,"scheduling_latency_s":87.722928}2121gitlab # [ 436.700697] nixbot[3674]: INFO: ::ffff:127.0.0.1:45954 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2122gitlab # [ 436.779914] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.629Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8c6d251164135c862f970f91","created_at":"2026-09-03T10:18:51.686Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-1a88e0644a674ae4","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:51.723Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8c6d251164135c862f970f91: done: 0.182932 sec","job_status":"done","queue_duration_s":87.722883,"scheduling_latency_s":87.722928,"redis_calls":16,"redis_duration_s":0.006274,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003165,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000808,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00074596,"duration_s":0.182932,"completed_at":"2026-09-03T10:20:19.629Z","load_balancing_strategy":"primary","db_duration_s":0.04635,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2123gitlab # [ 436.861308] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.632Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"01b3a0c75dd903ddbe61e9f1","created_at":"2026-09-03T10:18:51.725Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-1a88e0644a674ae4","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:51.740Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-01b3a0c75dd903ddbe61e9f1: start","job_status":"start","queue_duration_s":87.891658,"scheduling_latency_s":87.89169}2124gitlab # [ 437.083878] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.932Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"01b3a0c75dd903ddbe61e9f1","created_at":"2026-09-03T10:18:51.725Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-1a88e0644a674ae4","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:51.740Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-01b3a0c75dd903ddbe61e9f1: done: 0.300529 sec","job_status":"done","queue_duration_s":87.891658,"scheduling_latency_s":87.89169,"redis_calls":21,"redis_duration_s":0.007616,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003808,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001997,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.235102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005073,"duration_s":0.300529,"completed_at":"2026-09-03T10:20:19.932Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 437.184748] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:19.935Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ab7a0d177e22d961d5efde63","created_at":"2026-09-03T10:18:51.933Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-1a88e0644a674ae4","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:51.942Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab7a0d177e22d961d5efde63: start","job_status":"start","queue_duration_s":87.992893,"scheduling_latency_s":87.992915}2126gitlab # [ 437.320641] gitaly[958]: time="2026-09-03T10:20:20.168Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:20.168" grpc.time_ms=0.693 pid=958 span.kind=server system=grpc2127gitlab # [ 437.377111] gitaly[958]: time="2026-09-03T10:20:20.227Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:20.226" grpc.time_ms=0.824 pid=958 span.kind=server system=grpc2128gitlab # [ 437.422778] gitaly[958]: time="2026-09-03T10:20:20.272Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:20.272" grpc.time_ms=0.514 pid=958 span.kind=server system=grpc2129gitlab # [ 437.451590] gitaly[958]: time="2026-09-03T10:20:20.284Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 diskcache=ea9c771f-9f00-4a63-9e9b-5dbc2790035f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:20:50.110" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:20.111" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2130gitlab # [ 437.473785] gitaly[958]: time="2026-09-03T10:20:20.285Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=192 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7530 command.oublock=0 command.real_time_ms=165 command.spawn_token_fork_ms=1 command.system_time_ms=126 command.user_time_ms=66 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:20:50.110" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:20.111" grpc.time_ms=173.433 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2131gitlab # [ 437.634688] gitaly[958]: time="2026-09-03T10:20:20.484Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:20:50.482" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:20.482" grpc.time_ms=1.602 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2132gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2133gitlab # [ 437.694787] gitaly[958]: time="2026-09-03T10:20:20.543Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEHHS483QG6E4QTJMPCV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:50.540" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:20.541" grpc.time_ms=1.251 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2134gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.07 seconds)2135gitlab # [ 437.733501] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:20.582Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ab7a0d177e22d961d5efde63","created_at":"2026-09-03T10:18:51.933Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-1a88e0644a674ae4","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:18:51.942Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab7a0d177e22d961d5efde63: done: 0.647182 sec","job_status":"done","queue_duration_s":87.992893,"scheduling_latency_s":87.992915,"gitaly_calls":3,"gitaly_duration_s":0.185965,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.036427,"redis_read_bytes":457,"redis_write_bytes":28001,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001664,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001641,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.012621,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005332,"redis_queues_read_bytes":36,"redis_queues_write_bytes":14495,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.007827,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000709,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001267,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003702,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.09,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.032,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.037,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.047,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.341565,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.56171428,"duration_s":0.647182,"completed_at":"2026-09-03T10:20:20.582Z","load_balancing_strategy":"primary","db_duration_s":0.107982,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2136gitlab # [ 437.838632] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:20.585Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"687ad358637c7a52dcd35d60","created_at":"2026-09-03T10:20:17.587Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:17.617Z","job_size_bytes":46,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-687ad358637c7a52dcd35d60: start","job_status":"start","queue_duration_s":2.967911,"scheduling_latency_s":2.967961}2137gitlab # [ 437.856509] nixbot[3674]: INFO: ::ffff:127.0.0.1:45964 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2138gitlab # [ 437.858515] gitaly[958]: time="2026-09-03T10:20:20.689Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-03T16:20:20.677" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:20.678" grpc.time_ms=10.312 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2139gitlab # [ 437.876218] gitaly[958]: time="2026-09-03T10:20:20.726Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-03T10:20:50.707" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:20.707" grpc.time_ms=17.911 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2140gitlab # [ 437.933972] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:20.783Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"687ad358637c7a52dcd35d60","created_at":"2026-09-03T10:20:17.587Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:17.617Z","job_size_bytes":46,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-687ad358637c7a52dcd35d60: done: 0.197254 sec","job_status":"done","queue_duration_s":2.967911,"scheduling_latency_s":2.967961,"gitaly_calls":2,"gitaly_duration_s":0.035632,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011918999999999999,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002454,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.001159,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002619,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004493,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.128482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005873,"duration_s":0.197254,"completed_at":"2026-09-03T10:20:20.782Z","load_balancing_strategy":"primary","db_duration_s":0.019342,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2141gitlab # [ 438.024346] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:20.785Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0a34434e5f3e074bc18102b6","created_at":"2026-09-03T10:20:17.646Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:17.655Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0a34434e5f3e074bc18102b6: start","job_status":"start","queue_duration_s":3.129803,"scheduling_latency_s":3.129831}2142gitlab # [ 438.046996] gitaly[958]: time="2026-09-03T10:20:20.858Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E diskcache=55a0f7f0-a1ed-4d3e-8d46-6e95b554bade grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:20:20.822" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:20.822" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2143gitlab # [ 438.059934] gitaly[958]: time="2026-09-03T10:20:20.858Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=23 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=39 command.spawn_token_fork_ms=4 command.system_time_ms=23 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:20:20.822" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:20.822" grpc.time_ms=35.743 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2144gitlab # [ 438.099736] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:20.948Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0a34434e5f3e074bc18102b6","created_at":"2026-09-03T10:20:17.646Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:17.655Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0a34434e5f3e074bc18102b6: done: 0.16317 sec","job_status":"done","queue_duration_s":3.129803,"scheduling_latency_s":3.129831,"gitaly_calls":1,"gitaly_duration_s":0.039237,"redis_calls":15,"redis_duration_s":0.006226000000000001,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000586,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000917,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002333,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.045,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11834592000000001,"duration_s":0.16317,"completed_at":"2026-09-03T10:20:20.948Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032822,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 438.158508] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:20.951Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"76814eb7ef3e3a4d9fbef939","created_at":"2026-09-03T10:20:17.849Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:17.852Z","job_size_bytes":23,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-76814eb7ef3e3a4d9fbef939: start","job_status":"start","queue_duration_s":3.098961,"scheduling_latency_s":3.099021}2146gitlab # [ 438.174063] gitaly[958]: time="2026-09-03T10:20:21.023Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-03T10:20:51.007" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:21.007" grpc.time_ms=15.736 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2147gitlab # [ 438.195232] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:21.043Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"76814eb7ef3e3a4d9fbef939","created_at":"2026-09-03T10:20:17.849Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:17.852Z","job_size_bytes":23,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-76814eb7ef3e3a4d9fbef939: done: 0.092117 sec","job_status":"done","queue_duration_s":3.098961,"scheduling_latency_s":3.099021,"gitaly_calls":1,"gitaly_duration_s":0.019444,"redis_calls":16,"redis_duration_s":0.006328,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000453,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.00297,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.092117,"completed_at":"2026-09-03T10:20:21.043Z","load_balancing_strategy":"primary","db_duration_s":0.005413,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2148gitlab # [ 438.239706] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:21.047Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"6f97de80da59369d7e57ba74","created_at":"2026-09-03T10:20:18.618Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a1e91260ab976970553ca9960047c15b3afd6156bba40711e7cae04c26e008c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:18.630Z","job_size_bytes":467,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-6f97de80da59369d7e57ba74: start","job_status":"start","queue_duration_s":2.416797,"scheduling_latency_s":2.416844}2149gitlab # [ 438.258254] gitaly[958]: time="2026-09-03T10:20:21.108Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:51.106" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:21.106" grpc.time_ms=1.04 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2150gitlab # [ 438.280332] gitaly[958]: time="2026-09-03T10:20:21.130Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:20:31.001" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:21.112" grpc.time_ms=17.533 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2151gitlab # [ 438.481816] gitaly[958]: time="2026-09-03T10:20:21.330Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:20:51.328" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:21.329" grpc.time_ms=1.261 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2152gitlab # [ 438.520438] gitaly[958]: time="2026-09-03T10:20:21.339Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=32 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-03T10:20:51.334" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:21.335" grpc.time_ms=3.859 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2153gitlab # [ 438.607211] gitaly[958]: time="2026-09-03T10:20:21.456Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:31.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:21.439" grpc.time_ms=17.495 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2154gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2155gitlab # [ 438.810385] nixbot[3674]: INFO: ::ffff:127.0.0.1:45972 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2156gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.07 seconds)2157gitlab # [ 439.366915] gitaly[958]: time="2026-09-03T10:20:22.216Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=2 command.system_time_ms=17 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-03T10:20:52.189" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:22.189" grpc.time_ms=27.002 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2158gitlab # [ 439.387354] gitaly[958]: time="2026-09-03T10:20:22.237Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEB3QRGGGWE8JHCDQQ71E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-03T10:20:52.222" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:22.222" grpc.time_ms=14.245 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2159gitlab # [ 439.450844] sidekiq-cluster[3622]: {"severity":"WARN","time":"2026-09-03T10:20:22.299Z","class":"CreatePipelineWorker","correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"dc6d2c375b1a031f8f5579a9741b23a7183f1fcc","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"dc6d2c375b1a031f8f5579a9741b23a7183f1fcc","ref":"refs/heads/master"},"retry":0}2160gitlab # [ 439.467503] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:22.316Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"6f97de80da59369d7e57ba74","created_at":"2026-09-03T10:20:18.618Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a1e91260ab976970553ca9960047c15b3afd6156bba40711e7cae04c26e008c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:18.630Z","job_size_bytes":467,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-6f97de80da59369d7e57ba74: done: 1.268802 sec","job_status":"done","queue_duration_s":2.416797,"scheduling_latency_s":2.416844,"gitaly_calls":7,"gitaly_duration_s":0.105245,"redis_calls":51,"redis_duration_s":0.015421,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.00116,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.006993,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002148,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003442,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001678,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.077975,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062832,"duration_s":1.268802,"completed_at":"2026-09-03T10:20:22.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084031,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2161gitlab # [ 439.542539] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:22.319Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"7aa16dc8-fa81-4fe4-bb97-d10143599ba2\"}"],"class":"WebHookWorker","jid":"7d1dc4e42e95bff429980dcc","created_at":"2026-09-03T10:20:18.857Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3eb07efd4888367347ba51c35b456910ed8723091cc35b9358e022b0a266f538","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:18.864Z","job_size_bytes":1751,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7d1dc4e42e95bff429980dcc: start","job_status":"start","queue_duration_s":3.455059,"scheduling_latency_s":3.455093}2162gitlab # [ 439.693113] nixbot[3674]: INFO: ::ffff:127.0.0.1:45988 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2163gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2164gitlab # [ 439.734578] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:22.583Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"7aa16dc8-fa81-4fe4-bb97-d10143599ba2\"}"],"class":"WebHookWorker","jid":"7d1dc4e42e95bff429980dcc","created_at":"2026-09-03T10:20:18.857Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-32d3796b5bec4969","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3eb07efd4888367347ba51c35b456910ed8723091cc35b9358e022b0a266f538","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:18.864Z","job_size_bytes":1751,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7d1dc4e42e95bff429980dcc: done: 0.263949 sec","job_status":"done","queue_duration_s":3.455059,"scheduling_latency_s":3.455093,"redis_calls":19,"redis_duration_s":0.006859,"redis_read_bytes":32,"redis_write_bytes":5365,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001712,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000723,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002973,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001451,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09918421999998372,"cpu_s":0.140515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066234,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.263949,"completed_at":"2026-09-03T10:20:22.583Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2165gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.06 seconds)2166gitlab # [ 439.857084] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:22.586Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8c6062d22f9fb8de558684b9","created_at":"2026-09-03T10:20:19.098Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-edcf655f045145d3","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0a34434e5f3e074bc18102b6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:19.103Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8c6062d22f9fb8de558684b9: start","job_status":"start","queue_duration_s":3.482679,"scheduling_latency_s":3.482712}2167gitlab # [ 439.883324] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:22.671Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8c6062d22f9fb8de558684b9","created_at":"2026-09-03T10:20:19.098Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-edcf655f045145d3","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0a34434e5f3e074bc18102b6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:19.103Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8c6062d22f9fb8de558684b9: done: 0.085357 sec","job_status":"done","queue_duration_s":3.482679,"scheduling_latency_s":3.482712,"gitaly_calls":1,"gitaly_duration_s":0.018191,"redis_calls":12,"redis_duration_s":0.004901,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001806,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001204,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04867566,"duration_s":0.085357,"completed_at":"2026-09-03T10:20:22.671Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 439.941747] gitaly[958]: time="2026-09-03T10:20:22.637Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT diskcache=47b97b5c-75d0-45f9-95d7-2488b4308c41 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:20:22.623" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:22.624" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2169gitlab # [ 439.951787] gitaly[958]: time="2026-09-03T10:20:22.638Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-03T16:20:22.623" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:22.624" grpc.time_ms=13.544 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2170gitlab # [ 439.965062] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:22.675Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"bc72531e227b8e8ea9676788","created_at":"2026-09-03T10:20:19.325Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-edcf655f045145d3","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0bf7b686748b31aff70974fc827c466a0c5038534b7cdb6ee4d74b55d32a61d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:19.331Z","job_size_bytes":467,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-bc72531e227b8e8ea9676788: start","job_status":"start","queue_duration_s":3.344473,"scheduling_latency_s":3.344506}2171gitlab # [ 439.983800] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:22 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1702172gitlab # [ 439.986032] nixbot[3674]: INFO: ::ffff:127.0.0.1:46000 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2173gitlab # [ 439.987894] gitaly[958]: time="2026-09-03T10:20:22.724Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:52.722" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:22.723" grpc.time_ms=1.053 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2174gitlab # [ 440.000292] gitaly[958]: time="2026-09-03T10:20:22.739Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:20:32.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:22.728" grpc.time_ms=11.051 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2175gitlab # [ 440.013066] gitaly[958]: time="2026-09-03T10:20:22.776Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHB6EXP26D44NDD6TX1XA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:22.764" grpc.time_ms=11.21 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2176gitlab # [ 440.026412] gitlab-workhorse[893]: time="2026-09-03T10:20:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672177gitlab # [ 440.028251] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 802178gitlab # [ 440.030203] gitaly[958]: time="2026-09-03T10:20:22.792Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:20:52.089" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:22.790" grpc.time_ms=1.72 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2179gitlab # [ 440.042480] gitaly[958]: time="2026-09-03T10:20:22.802Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=231 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-03T10:20:52.096" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:22.797" grpc.time_ms=4.931 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2180gitlab # [ 440.058330] gitaly[958]: time="2026-09-03T10:20:22.856Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:32.117" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:22.838" grpc.time_ms=17.384 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2181gitlab # [ 440.071592] gitaly[958]: time="2026-09-03T10:20:22.858Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHBBVQE2F08F4KFPTHA4Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:22.848" pid=958 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2182gitlab # [ 440.081863] gitaly[958]: time="2026-09-03T10:20:22.858Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHBBVQE2F08F4KFPTHA4Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:22.848" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2183gitlab # [ 440.094800] nixbot[3674]: {"ts": "2026-09-03T10:20:22.908Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2184gitlab # [ 440.097296] gitaly[958]: time="2026-09-03T10:20:22.859Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHBBVQE2F08F4KFPTHA4Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:22.848" grpc.time_ms=10.715 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2185gitlab # [ 440.187788] gitaly[958]: time="2026-09-03T10:20:23.037Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:53.035" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:23.036" grpc.time_ms=1.045 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2186gitlab # [ 440.398544] gitaly[958]: time="2026-09-03T10:20:23.248Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:33.150" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:23.230" grpc.time_ms=17.557 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2187gitlab # [ 440.588171] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:23 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 5052188gitlab # [ 440.595445] nixbot[3674]: {"ts": "2026-09-03T10:20:23.439Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc \"HTTP/1.1 201 Created\""}2189gitlab # [ 440.690401] gitaly[958]: time="2026-09-03T10:20:23.540Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-03T10:20:53.011" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:23.511" grpc.time_ms=28.157 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2190gitlab # [ 440.729227] gitaly[958]: time="2026-09-03T10:20:23.563Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M1KCEGB0170C7ZY19CZVYXFT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-03T10:20:53.043" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:23.543" grpc.time_ms=19.516 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2191gitlab # [ 440.751898] sidekiq-cluster[3622]: {"severity":"WARN","time":"2026-09-03T10:20:23.592Z","class":"CreatePipelineWorker","correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"dc6d2c375b1a031f8f5579a9741b23a7183f1fcc","after":"05c5fe8317a89e596d33e6555010cf0e802e5771","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"05c5fe8317a89e596d33e6555010cf0e802e5771","ref":"refs/heads/master"},"retry":0}2192gitlab # [ 440.762082] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.605Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"bc72531e227b8e8ea9676788","created_at":"2026-09-03T10:20:19.325Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-edcf655f045145d3","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0bf7b686748b31aff70974fc827c466a0c5038534b7cdb6ee4d74b55d32a61d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:19.331Z","job_size_bytes":467,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-bc72531e227b8e8ea9676788: done: 0.929722 sec","job_status":"done","queue_duration_s":3.344473,"scheduling_latency_s":3.344506,"gitaly_calls":7,"gitaly_duration_s":0.107809,"redis_calls":33,"redis_duration_s":0.008672000000000001,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000937,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003303,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.785803,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.929722,"completed_at":"2026-09-03T10:20:23.605Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034074,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2193gitlab # [ 440.820391] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.609Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"b7e105a1-c372-4ab5-adf5-945ae7d4bd93\"}"],"class":"WebHookWorker","jid":"70b98e4d928e6093c2e8f470","created_at":"2026-09-03T10:20:19.363Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-edcf655f045145d3","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:50809eb7b07d9e018c13e50feac308d8df94dcfc2a044aadbb1dec35f350fc44","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:19.368Z","job_size_bytes":1749,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-70b98e4d928e6093c2e8f470: start","job_status":"start","queue_duration_s":4.241415,"scheduling_latency_s":4.241457}2194gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2195gitlab # [ 440.843832] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.689Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"b7e105a1-c372-4ab5-adf5-945ae7d4bd93\"}"],"class":"WebHookWorker","jid":"70b98e4d928e6093c2e8f470","created_at":"2026-09-03T10:20:19.363Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-edcf655f045145d3","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:50809eb7b07d9e018c13e50feac308d8df94dcfc2a044aadbb1dec35f350fc44","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:19.368Z","job_size_bytes":1749,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-70b98e4d928e6093c2e8f470: done: 0.079869 sec","job_status":"done","queue_duration_s":4.241415,"scheduling_latency_s":4.241457,"redis_calls":14,"redis_duration_s":0.007523,"redis_read_bytes":26,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000545,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003028,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00395,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.014660079999998743,"cpu_s":0.047539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.079869,"completed_at":"2026-09-03T10:20:23.689Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196gitlab # [ 440.894490] nixbot[3674]: INFO: ::ffff:127.0.0.1:46014 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2197gitlab # [ 440.895751] gitaly[958]: time="2026-09-03T10:20:23.633Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHC0RGE2PVS7VHS53FAAJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:53.631" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:23.632" grpc.time_ms=1.21 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2198gitlab # [ 440.908653] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.696Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fddfd9d9cf0ede3b64813831","created_at":"2026-09-03T10:20:20.092Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.097Z","job_size_bytes":23,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fddfd9d9cf0ede3b64813831: start","job_status":"start","queue_duration_s":3.598669,"scheduling_latency_s":3.598703}2199gitlab # [ 440.924898] gitaly[958]: time="2026-09-03T10:20:23.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHC0RGE2PVS7VHS53FAAJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:33.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:23.727" grpc.time_ms=11.737 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200gitlab # [ 440.938014] nixbot[3674]: INFO: ::ffff:127.0.0.1:34992 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2201gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.07 seconds)2202gitlab # [ 440.967351] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:23 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3682203gitlab # [ 440.970429] nixbot[3674]: {"ts": "2026-09-03T10:20:23.818Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc \"HTTP/1.1 201 Created\""}2204gitlab # [ 440.988781] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.837Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fddfd9d9cf0ede3b64813831","created_at":"2026-09-03T10:20:20.092Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.097Z","job_size_bytes":23,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fddfd9d9cf0ede3b64813831: done: 0.140849 sec","job_status":"done","queue_duration_s":3.598669,"scheduling_latency_s":3.598703,"redis_calls":10,"redis_duration_s":0.0040349999999999995,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000774,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005277,"duration_s":0.140849,"completed_at":"2026-09-03T10:20:23.837Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032206,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2205gitlab # [ 441.033939] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.838Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0866ec7e106bc5b3299843a1","created_at":"2026-09-03T10:20:20.294Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.307Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0866ec7e106bc5b3299843a1: start","job_status":"start","queue_duration_s":3.531861,"scheduling_latency_s":3.53188}2206gitlab # [ 441.050130] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.878Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0866ec7e106bc5b3299843a1","created_at":"2026-09-03T10:20:20.294Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.307Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0866ec7e106bc5b3299843a1: done: 0.039962 sec","job_status":"done","queue_duration_s":3.531861,"scheduling_latency_s":3.53188,"redis_calls":10,"redis_duration_s":0.001462,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027098,"duration_s":0.039962,"completed_at":"2026-09-03T10:20:23.878Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207gitlab # [ 441.096353] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:23 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1322208gitlab # [ 441.098423] gitlab-workhorse[893]: time="2026-09-03T10:20:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672209gitlab # [ 441.100257] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 772210gitlab # [ 441.102399] gitaly[958]: time="2026-09-03T10:20:23.848Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHC94NSBKW2S6PFW53DTQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:23.832" grpc.time_ms=16.647 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2211gitlab # [ 441.120262] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.880Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"233eeff8966b804ff6e34e32","created_at":"2026-09-03T10:20:20.336Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.347Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-233eeff8966b804ff6e34e32: start","job_status":"start","queue_duration_s":3.532863,"scheduling_latency_s":3.53288}2212gitlab # [ 441.135834] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.922Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"233eeff8966b804ff6e34e32","created_at":"2026-09-03T10:20:20.336Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.347Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-233eeff8966b804ff6e34e32: done: 0.042218 sec","job_status":"done","queue_duration_s":3.532863,"scheduling_latency_s":3.53288,"redis_calls":10,"redis_duration_s":0.002046,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036374,"duration_s":0.042218,"completed_at":"2026-09-03T10:20:23.922Z","load_balancing_strategy":"primary","db_duration_s":0.00858,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213gitlab # [ 441.178773] gitaly[958]: time="2026-09-03T10:20:23.904Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCCGH58XQC92DBBMF7QC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:53.102" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:23.902" grpc.time_ms=1.568 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2214gitlab # [ 441.191469] gitaly[958]: time="2026-09-03T10:20:23.938Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCDQA19Z4XN7DMAZGV5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:23.927" pid=958 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2215gitlab # [ 441.201772] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.923Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"488ff51eb89f48aaf66b0c07","created_at":"2026-09-03T10:20:20.375Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.385Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-488ff51eb89f48aaf66b0c07: start","job_status":"start","queue_duration_s":3.537708,"scheduling_latency_s":3.537724}2216gitlab # [ 441.218333] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.978Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"488ff51eb89f48aaf66b0c07","created_at":"2026-09-03T10:20:20.375Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.385Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-488ff51eb89f48aaf66b0c07: done: 0.0548 sec","job_status":"done","queue_duration_s":3.537708,"scheduling_latency_s":3.537724,"redis_calls":13,"redis_duration_s":0.006201,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002278,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.003266,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004989,"duration_s":0.0548,"completed_at":"2026-09-03T10:20:23.978Z","load_balancing_strategy":"primary","db_duration_s":0.008397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2217gitlab # [ 441.262556] gitaly[958]: time="2026-09-03T10:20:23.938Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCDQA19Z4XN7DMAZGV5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:23.927" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2218gitlab # [ 441.275716] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2162219gitlab # [ 441.278338] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 972220gitlab # [ 441.280385] nixbot[3674]: {"ts": "2026-09-03T10:20:24.038Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/dc6d2c375b1a031f8f5579a9741b23a7183f1fcc \"HTTP/1.1 201 Created\""}2221gitlab # [ 441.283070] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:23.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3bf2486241d79e6e72dcb040","created_at":"2026-09-03T10:20:20.387Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.395Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3bf2486241d79e6e72dcb040: start","job_status":"start","queue_duration_s":3.584723,"scheduling_latency_s":3.584742}2222gitlab # [ 441.299895] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.026Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"3bf2486241d79e6e72dcb040","created_at":"2026-09-03T10:20:20.387Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.395Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-3bf2486241d79e6e72dcb040: done: 0.045986 sec","job_status":"done","queue_duration_s":3.584723,"scheduling_latency_s":3.584742,"redis_calls":10,"redis_duration_s":0.0022180000000000004,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024304,"duration_s":0.045986,"completed_at":"2026-09-03T10:20:24.026Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223gitlab # [ 441.344190] gitaly[958]: time="2026-09-03T10:20:23.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCDQA19Z4XN7DMAZGV5M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:23.927" grpc.time_ms=11.559 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2224gitlab # [ 441.358327] gitlab-workhorse[893]: time="2026-09-03T10:20:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672225gitlab # [ 441.360105] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 812226gitlab # [ 441.361901] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.027Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3cc24ae53aeddf9bcae6af1f","created_at":"2026-09-03T10:20:20.396Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.405Z","job_size_bytes":19,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3cc24ae53aeddf9bcae6af1f: start","job_status":"start","queue_duration_s":3.622288,"scheduling_latency_s":3.622306}2227gitlab # [ 441.377918] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.072Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3cc24ae53aeddf9bcae6af1f","created_at":"2026-09-03T10:20:20.396Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.405Z","job_size_bytes":19,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3cc24ae53aeddf9bcae6af1f: done: 0.044694 sec","job_status":"done","queue_duration_s":3.622288,"scheduling_latency_s":3.622306,"redis_calls":11,"redis_duration_s":0.0042,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025614,"duration_s":0.044694,"completed_at":"2026-09-03T10:20:24.071Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2228gitlab # [ 441.422440] gitaly[958]: time="2026-09-03T10:20:23.962Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCCGH58XQC92DBBMF7QC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:33.233" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:23.954" grpc.time_ms=7.969 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2229gitlab # [ 441.435900] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 722230gitlab # [ 441.438083] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.073Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3823349ddb29ca3a672c7e83","created_at":"2026-09-03T10:20:20.411Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.418Z","job_size_bytes":66,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3823349ddb29ca3a672c7e83: start","job_status":"start","queue_duration_s":3.655129,"scheduling_latency_s":3.655148}2231gitlab # [ 441.455317] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.117Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3823349ddb29ca3a672c7e83","created_at":"2026-09-03T10:20:20.411Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.418Z","job_size_bytes":66,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3823349ddb29ca3a672c7e83: done: 0.043968 sec","job_status":"done","queue_duration_s":3.655129,"scheduling_latency_s":3.655148,"redis_calls":10,"redis_duration_s":0.0038929999999999998,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004355,"duration_s":0.043968,"completed_at":"2026-09-03T10:20:24.117Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2232gitlab # [ 441.500284] gitaly[958]: time="2026-09-03T10:20:24.073Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=10 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCH8K5RRJBKPP7MPJ3J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.058" grpc.time_ms=15.492 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2233gitlab # [ 441.514425] gitaly[958]: time="2026-09-03T10:20:24.156Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCMBFZJZYWVEAE79CW8X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.146" pid=958 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2234gitlab # [ 441.524791] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.121Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"148a1faac0206f462d3c0297","created_at":"2026-09-03T10:20:20.419Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.426Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-148a1faac0206f462d3c0297: start","job_status":"start","queue_duration_s":3.695157,"scheduling_latency_s":3.695174}2235gitlab # [ 441.540599] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.192Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"148a1faac0206f462d3c0297","created_at":"2026-09-03T10:20:20.419Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.426Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-148a1faac0206f462d3c0297: done: 0.071518 sec","job_status":"done","queue_duration_s":3.695157,"scheduling_latency_s":3.695174,"redis_calls":10,"redis_duration_s":0.00262,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000512,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":10,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003919,"duration_s":0.071518,"completed_at":"2026-09-03T10:20:24.192Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236gitlab # [ 441.586448] gitlab-workhorse[893]: time="2026-09-03T10:20:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162237gitlab # [ 441.588278] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 842238gitlab # [ 441.590146] gitaly[958]: time="2026-09-03T10:20:24.156Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCMBFZJZYWVEAE79CW8X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.146" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2239gitlab # [ 441.603895] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.195Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"e6c0b472c61aabfb47d7459e","created_at":"2026-09-03T10:20:20.436Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.442Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e6c0b472c61aabfb47d7459e: start","job_status":"start","queue_duration_s":3.753443,"scheduling_latency_s":3.753464}2240gitlab # [ 441.621389] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.238Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"e6c0b472c61aabfb47d7459e","created_at":"2026-09-03T10:20:20.436Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.442Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e6c0b472c61aabfb47d7459e: done: 0.042533 sec","job_status":"done","queue_duration_s":3.753443,"scheduling_latency_s":3.753464,"redis_calls":10,"redis_duration_s":0.002128,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026488,"duration_s":0.042533,"completed_at":"2026-09-03T10:20:24.237Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 441.665916] gitaly[958]: time="2026-09-03T10:20:24.156Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCMBFZJZYWVEAE79CW8X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.146" grpc.time_ms=9.87 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2242gitlab # [ 441.680396] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.240Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"561088512455ead5f0863983","created_at":"2026-09-03T10:20:20.442Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.447Z","job_size_bytes":6,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-561088512455ead5f0863983: start","job_status":"start","queue_duration_s":3.792666,"scheduling_latency_s":3.792682}2243gitlab # [ 441.696342] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.274Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"561088512455ead5f0863983","created_at":"2026-09-03T10:20:20.442Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.447Z","job_size_bytes":6,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-561088512455ead5f0863983: done: 0.033797 sec","job_status":"done","queue_duration_s":3.792666,"scheduling_latency_s":3.792682,"redis_calls":10,"redis_duration_s":0.001384,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000329,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023914,"duration_s":0.033797,"completed_at":"2026-09-03T10:20:24.274Z","load_balancing_strategy":"primary","db_duration_s":0.011086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2244gitlab # [ 441.740449] gitaly[958]: time="2026-09-03T10:20:24.264Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCR15T1K32JKPB7JE2P6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.254" grpc.time_ms=9.8 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2245gitlab # [ 441.753515] gitaly[958]: time="2026-09-03T10:20:24.353Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCTDXM8R00W4SZ4PZB20 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.328" pid=958 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2246gitlab # [ 441.764237] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.275Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"64c96236998444fcf31d1a6f","created_at":"2026-09-03T10:20:20.448Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.453Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-64c96236998444fcf31d1a6f: start","job_status":"start","queue_duration_s":3.822232,"scheduling_latency_s":3.822249}2247gitlab # [ 441.780106] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.320Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"64c96236998444fcf31d1a6f","created_at":"2026-09-03T10:20:20.448Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.453Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-64c96236998444fcf31d1a6f: done: 0.044972 sec","job_status":"done","queue_duration_s":3.822232,"scheduling_latency_s":3.822249,"redis_calls":10,"redis_duration_s":0.001513,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025732,"duration_s":0.044972,"completed_at":"2026-09-03T10:20:24.320Z","load_balancing_strategy":"primary","db_duration_s":0.013671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 441.824178] gitaly[958]: time="2026-09-03T10:20:24.353Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCTDXM8R00W4SZ4PZB20 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.328" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2249gitlab # [ 441.837838] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.322Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f5e300b295b328da9bdb2056","created_at":"2026-09-03T10:20:20.462Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.468Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f5e300b295b328da9bdb2056: start","job_status":"start","queue_duration_s":3.854212,"scheduling_latency_s":3.854229}2250gitlab # [ 441.853948] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.408Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f5e300b295b328da9bdb2056","created_at":"2026-09-03T10:20:20.462Z","trace_propagation_headers":{"sentry-trace":"f284457c1c574c7d88c05e07325edc35-4883027b7b0f4a95","baggage":"sentry-trace_id=f284457c1c574c7d88c05e07325edc35,sentry-release=Unknown"},"correlation_id":"01M1KCEHHS483QG6E4QTJMPCV1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.468Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f5e300b295b328da9bdb2056: done: 0.086123 sec","job_status":"done","queue_duration_s":3.854212,"scheduling_latency_s":3.854229,"redis_calls":14,"redis_duration_s":0.003086,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032378,"duration_s":0.086123,"completed_at":"2026-09-03T10:20:24.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251gitlab # [ 441.899625] gitaly[958]: time="2026-09-03T10:20:24.354Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCTDXM8R00W4SZ4PZB20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-03T10:20:24.328" grpc.time_ms=25.643 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc2252gitlab # [ 441.915323] nixbot[3674]: {"ts": "2026-09-03T10:20:24.729Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 \"HTTP/1.1 201 Created\""}2253gitlab # [ 441.918268] nixbot[3674]: {"ts": "2026-09-03T10:20:24.740Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-05c5fe8317a8-4"}2254gitlab # [ 441.920583] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.410Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e658ace61409d2337e0be90e","created_at":"2026-09-03T10:20:20.744Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-72b8660e75784d6e","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.756Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e658ace61409d2337e0be90e: start","job_status":"start","queue_duration_s":3.653485,"scheduling_latency_s":3.653503}2255gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771'2256gitlab # [ 441.936788] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.475Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e658ace61409d2337e0be90e","created_at":"2026-09-03T10:20:20.744Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-72b8660e75784d6e","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:20.756Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e658ace61409d2337e0be90e: done: 0.065489 sec","job_status":"done","queue_duration_s":3.653485,"scheduling_latency_s":3.653503,"redis_calls":19,"redis_duration_s":0.009779,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0012,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.003026,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003631,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01319212,"duration_s":0.065489,"completed_at":"2026-09-03T10:20:24.475Z","load_balancing_strategy":"primary","db_duration_s":0.009564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab # [ 441.979062] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3382258gitlab # [ 441.981486] gitaly[958]: time="2026-09-03T10:20:24.458Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:54.455" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:24.456" grpc.time_ms=1.54 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2259gitlab # [ 441.993907] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.477Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7d1dc4e42e95bff429980dcc"],"class":"WebHooks::LogExecutionWorker","jid":"2c8c29c0aa197a4351888bb7","created_at":"2026-09-03T10:20:22.548Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-50ba48c472c64bff","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a27d05c3e73a4159695829e843fa1277ae5d9b8b7a7f1b1c9e8fe7add9a6cd6c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:22.561Z","job_size_bytes":2592,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-2c8c29c0aa197a4351888bb7: start","job_status":"start","queue_duration_s":1.916178,"scheduling_latency_s":1.916198}2260gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771', in 0.09 seconds)2261(finished: subtest: push delivers the webhook, in 95.68 seconds)2262subtest: eval and build post commit statuses2263gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2264gitlab # [ 442.010355] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.553Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7d1dc4e42e95bff429980dcc"],"class":"WebHooks::LogExecutionWorker","jid":"2c8c29c0aa197a4351888bb7","created_at":"2026-09-03T10:20:22.548Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-50ba48c472c64bff","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a27d05c3e73a4159695829e843fa1277ae5d9b8b7a7f1b1c9e8fe7add9a6cd6c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:22.561Z","job_size_bytes":2592,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-2c8c29c0aa197a4351888bb7: done: 0.075638 sec","job_status":"done","queue_duration_s":1.916178,"scheduling_latency_s":1.916198,"redis_calls":10,"redis_duration_s":0.001539,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000341,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002241,"duration_s":0.075638,"completed_at":"2026-09-03T10:20:24.553Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00977,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265gitlab # [ 442.056953] gitaly[958]: time="2026-09-03T10:20:24.476Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-03T10:20:54.467" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:24.467" grpc.time_ms=9.021 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2266gitlab # [ 442.070613] nixbot[3674]: INFO: ::ffff:127.0.0.1:35000 - "GET /api/repos/gitlab/root/test-flake/builds?commit=05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 200 OK2267gitlab # [ 442.072516] nixbot[3674]: {"ts": "2026-09-03T10:20:24.893Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-05c5fe8317a8-4", "argv0": "bwrap"}2268gitlab # [ 442.074882] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.554Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"586f455a323c2dd37f4f5e94","created_at":"2026-09-03T10:20:23.382Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-142c6a6a59c84f6f","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:23.389Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-586f455a323c2dd37f4f5e94: start","job_status":"start","queue_duration_s":1.165416,"scheduling_latency_s":1.165433}2269gitlab # [ 442.090921] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.808Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"586f455a323c2dd37f4f5e94","created_at":"2026-09-03T10:20:23.382Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-142c6a6a59c84f6f","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:23.389Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3622,"job_status":"rescheduled","message":"PipelineProcessWorker JID-586f455a323c2dd37f4f5e94: rescheduled"}2270gitlab # [ 442.106779] gitaly[958]: time="2026-09-03T10:20:24.499Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:34.214" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:24.484" grpc.time_ms=14.637 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2271gitlab # [ 442.120191] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.816Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"586f455a323c2dd37f4f5e94","created_at":"2026-09-03T10:20:23.382Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-142c6a6a59c84f6f","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:23.389Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-586f455a323c2dd37f4f5e94: done: 0.261645 sec","job_status":"done","queue_duration_s":1.165416,"scheduling_latency_s":1.165433,"gitaly_calls":3,"gitaly_duration_s":0.090398,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014049,"redis_read_bytes":151,"redis_write_bytes":18011,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000491,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002936,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003404,"redis_queues_read_bytes":29,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004688,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000204,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000499,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001336,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.135374,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.22270352,"duration_s":0.261645,"completed_at":"2026-09-03T10:20:24.816Z","load_balancing_strategy":"primary","db_duration_s":0.031219,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2272gitlab # [ 442.171587] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.818Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","70b98e4d928e6093c2e8f470"],"class":"WebHooks::LogExecutionWorker","jid":"4527cd8e2820dd65a3d6edf5","created_at":"2026-09-03T10:20:23.668Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-2a93ebef0a6f4563","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:efb485a8a0dcd74167cd9d66f84af872f39098066a74f0a60972276c6058728f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:23.674Z","job_size_bytes":2591,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4527cd8e2820dd65a3d6edf5: start","job_status":"start","queue_duration_s":1.143784,"scheduling_latency_s":1.14381}2273gitlab # [ 442.188425] gitaly[958]: time="2026-09-03T10:20:24.519Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-03T10:20:34.248" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:24.519" grpc.time_ms=0.266 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2274gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.19 seconds)2275gitlab # [ 442.199171] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.854Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","70b98e4d928e6093c2e8f470"],"class":"WebHooks::LogExecutionWorker","jid":"4527cd8e2820dd65a3d6edf5","created_at":"2026-09-03T10:20:23.668Z","trace_propagation_headers":{"sentry-trace":"c98672d8773349929e114580ce4990e8-2a93ebef0a6f4563","baggage":"sentry-trace_id=c98672d8773349929e114580ce4990e8,sentry-release=Unknown"},"correlation_id":"01M1KCEGB0170C7ZY19CZVYXFT","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:efb485a8a0dcd74167cd9d66f84af872f39098066a74f0a60972276c6058728f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:23.674Z","job_size_bytes":2591,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4527cd8e2820dd65a3d6edf5: done: 0.036257 sec","job_status":"done","queue_duration_s":1.143784,"scheduling_latency_s":1.14381,"redis_calls":7,"redis_duration_s":0.004787,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003162,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036257,"completed_at":"2026-09-03T10:20:24.854Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006473,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 442.245039] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.857Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"411af4d255402bf3be70ef84","created_at":"2026-09-03T10:20:24.454Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-98906349261145e9","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.459Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-411af4d255402bf3be70ef84: start","job_status":"start","queue_duration_s":0.3975,"scheduling_latency_s":0.397524}2277gitlab # [ 442.262324] nix-daemon[3902]: accepted connection from pid 4567, user nixbot2278gitlab # [ 442.263282] gitaly[958]: time="2026-09-03T10:20:24.529Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-03T10:20:34.253" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:24.523" grpc.time_ms=6.025 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2279gitlab # [ 442.277020] gitaly[958]: time="2026-09-03T10:20:24.582Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:24.581" grpc.time_ms=0.691 pid=958 span.kind=server system=grpc2280gitlab # [ 442.286708] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.966Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"411af4d255402bf3be70ef84","created_at":"2026-09-03T10:20:24.454Z","trace_propagation_headers":{"sentry-trace":"d513a6e1804b4267b3640dcd7be20fbf-98906349261145e9","baggage":"sentry-trace_id=d513a6e1804b4267b3640dcd7be20fbf,sentry-release=Unknown"},"correlation_id":"01M1KCEB3QRGGGWE8JHCDQQ71E","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.459Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-411af4d255402bf3be70ef84: done: 0.109425 sec","job_status":"done","queue_duration_s":0.3975,"scheduling_latency_s":0.397524,"redis_calls":11,"redis_duration_s":0.00711,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000419,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006127,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046428,"duration_s":0.109425,"completed_at":"2026-09-03T10:20:24.966Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02717,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281gitlab # [ 442.331050] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:24.970Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b3f9c8e8998d5cb987a60e61","created_at":"2026-09-03T10:20:24.619Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.621Z","job_size_bytes":23,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b3f9c8e8998d5cb987a60e61: start","job_status":"start","queue_duration_s":0.348697,"scheduling_latency_s":0.348721}2282gitlab # [ 442.347920] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1072283gitlab # [ 442.350695] gitaly[958]: time="2026-09-03T10:20:24.621Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:24.618" grpc.time_ms=2.799 pid=958 span.kind=server system=grpc2284gitlab # [ 442.360458] gitaly[958]: time="2026-09-03T10:20:24.629Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 diskcache=48a813e6-38d6-4db5-92fa-45fa7af966c2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:20:34.262" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:24.532" pid=958 ref=refs/keep-around/05c5fe8317a89e596d33e6555010cf0e802e5771 remote_ip="2001:db8:1::1" revision=05c5fe8317a89e596d33e6555010cf0e802e5771 span.kind=server system=grpc username=root2285gitlab # [ 442.371563] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.029Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b3f9c8e8998d5cb987a60e61","created_at":"2026-09-03T10:20:24.619Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.621Z","job_size_bytes":23,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b3f9c8e8998d5cb987a60e61: done: 0.058691 sec","job_status":"done","queue_duration_s":0.348697,"scheduling_latency_s":0.348721,"redis_calls":7,"redis_duration_s":0.007449000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006982,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058691,"completed_at":"2026-09-03T10:20:25.029Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00852,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2286gitlab # [ 442.413961] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.031Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bb2e5bfa97716e994a1dd87f","created_at":"2026-09-03T10:20:24.637Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-c4e21096ac794ad6","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.643Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bb2e5bfa97716e994a1dd87f: start","job_status":"start","queue_duration_s":0.387602,"scheduling_latency_s":0.387629}2287gitlab # [ 442.430618] gitaly[958]: time="2026-09-03T10:20:24.630Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=87 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5154 command.oublock=8 command.real_time_ms=92 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-03T10:20:34.262" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-03T10:20:24.532" grpc.time_ms=97.806 pid=958 ref=refs/keep-around/05c5fe8317a89e596d33e6555010cf0e802e5771 remote_ip="2001:db8:1::1" revision=05c5fe8317a89e596d33e6555010cf0e802e5771 span.kind=server system=grpc username=root2288gitlab # [ 442.446589] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.066Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bb2e5bfa97716e994a1dd87f","created_at":"2026-09-03T10:20:24.637Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-c4e21096ac794ad6","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.643Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bb2e5bfa97716e994a1dd87f: done: 0.035598 sec","job_status":"done","queue_duration_s":0.387602,"scheduling_latency_s":0.387629,"redis_calls":7,"redis_duration_s":0.002236,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001225,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035598,"completed_at":"2026-09-03T10:20:25.066Z","load_balancing_strategy":"primary","db_duration_s":0.004431,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2289gitlab # [ 442.488484] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.068Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c6dac7fddc5a183620854b7a","created_at":"2026-09-03T10:20:24.644Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-c4e21096ac794ad6","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.648Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c6dac7fddc5a183620854b7a: start","job_status":"start","queue_duration_s":0.419605,"scheduling_latency_s":0.419628}2290gitlab # [ 442.505397] gitaly[958]: time="2026-09-03T10:20:24.634Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:54.633" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:24.633" grpc.time_ms=0.722 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2291gitlab # [ 442.517788] gitaly[958]: time="2026-09-03T10:20:24.658Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:24.657" grpc.time_ms=0.66 pid=958 span.kind=server system=grpc2292gitlab # [ 442.527264] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.133Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c6dac7fddc5a183620854b7a","created_at":"2026-09-03T10:20:24.644Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-c4e21096ac794ad6","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.648Z","job_size_bytes":65,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c6dac7fddc5a183620854b7a: done: 0.065031 sec","job_status":"done","queue_duration_s":0.419605,"scheduling_latency_s":0.419628,"redis_calls":10,"redis_duration_s":0.002143,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065031,"completed_at":"2026-09-03T10:20:25.133Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2293gitlab # [ 442.569779] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.135Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"0616a0c6051c8fdb0442b355","created_at":"2026-09-03T10:20:24.709Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-c4e21096ac794ad6","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.713Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0616a0c6051c8fdb0442b355: start","job_status":"start","queue_duration_s":0.422041,"scheduling_latency_s":0.422081}2294gitlab # [ 442.585087] gitaly[958]: time="2026-09-03T10:20:24.678Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:24.678" grpc.time_ms=0.363 pid=958 span.kind=server system=grpc2295gitlab # [ 442.594434] gitaly[958]: time="2026-09-03T10:20:24.703Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-03T10:20:24.702" grpc.time_ms=0.364 pid=958 span.kind=server system=grpc2296gitlab # [ 442.604001] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.309Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"0616a0c6051c8fdb0442b355","created_at":"2026-09-03T10:20:24.709Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-c4e21096ac794ad6","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.713Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0616a0c6051c8fdb0442b355: done: 0.174488 sec","job_status":"done","queue_duration_s":0.422041,"scheduling_latency_s":0.422081,"gitaly_calls":2,"gitaly_duration_s":0.009569,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008956,"redis_read_bytes":80,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001451,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001431,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000446,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001965,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000361,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000635,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.125408,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1281577,"duration_s":0.174488,"completed_at":"2026-09-03T10:20:25.309Z","load_balancing_strategy":"primary","db_duration_s":0.038224,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2297gitlab # [ 442.654818] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.312Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d5693a11d504a95a84a06c3c","created_at":"2026-09-03T10:20:24.714Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.717Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d5693a11d504a95a84a06c3c: start","job_status":"start","queue_duration_s":0.594939,"scheduling_latency_s":0.594967}2298gitlab # [ 442.671328] gitaly[958]: time="2026-09-03T10:20:24.710Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G diskcache=8258dc04-8964-46ee-a51b-a0dce8471266 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:20:54.128" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:24.628" pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2299gitlab # [ 442.680853] gitaly[958]: time="2026-09-03T10:20:24.710Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=80 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7011 command.oublock=0 command.real_time_ms=77 command.spawn_token_fork_ms=1 command.system_time_ms=45 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-03T10:20:54.128" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:24.628" grpc.time_ms=82.197 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2300gitlab # [ 442.693414] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.348Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d5693a11d504a95a84a06c3c","created_at":"2026-09-03T10:20:24.714Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.717Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d5693a11d504a95a84a06c3c: done: 0.036281 sec","job_status":"done","queue_duration_s":0.594939,"scheduling_latency_s":0.594967,"redis_calls":7,"redis_duration_s":0.00204,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001462,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000578,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036281,"completed_at":"2026-09-03T10:20:25.348Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007703,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2301gitlab # [ 442.735013] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.350Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2a1ffb51c9c84307b053742b","created_at":"2026-09-03T10:20:24.722Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.726Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2a1ffb51c9c84307b053742b: start","job_status":"start","queue_duration_s":0.623911,"scheduling_latency_s":0.62394}2302gitlab # [ 442.751048] gitaly[958]: time="2026-09-03T10:20:24.765Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:20:54.063" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:24.763" grpc.time_ms=1.239 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2303gitlab # [ 442.763663] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.387Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2a1ffb51c9c84307b053742b","created_at":"2026-09-03T10:20:24.722Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.726Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2a1ffb51c9c84307b053742b: done: 0.03683 sec","job_status":"done","queue_duration_s":0.623911,"scheduling_latency_s":0.62394,"redis_calls":7,"redis_duration_s":0.002226,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001639,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000587,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03683,"completed_at":"2026-09-03T10:20:25.387Z","load_balancing_strategy":"primary","db_duration_s":0.008371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2304gitlab # [ 442.805682] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"cb8a355b358b66bd444ff436","created_at":"2026-09-03T10:20:24.733Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.735Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-cb8a355b358b66bd444ff436: start","job_status":"start","queue_duration_s":0.653861,"scheduling_latency_s":0.653887}2305gitlab # [ 442.821093] gitaly[958]: time="2026-09-03T10:20:24.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHBGMTZ4GCM0SWB690G1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:54.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:24.798" grpc.time_ms=0.674 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2306gitlab # [ 442.833566] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.459Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"cb8a355b358b66bd444ff436","created_at":"2026-09-03T10:20:24.733Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.735Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-cb8a355b358b66bd444ff436: done: 0.069885 sec","job_status":"done","queue_duration_s":0.653861,"scheduling_latency_s":0.653887,"redis_calls":10,"redis_duration_s":0.004606,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002474,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001373,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000759,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069885,"completed_at":"2026-09-03T10:20:25.459Z","load_balancing_strategy":"primary","db_duration_s":0.010444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2307gitlab # [ 442.877181] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.462Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"217b4c4f2cf493480a064b44","created_at":"2026-09-03T10:20:24.736Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.738Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-217b4c4f2cf493480a064b44: start","job_status":"start","queue_duration_s":0.724609,"scheduling_latency_s":0.724646}2308gitlab # [ 442.893596] gitaly[958]: time="2026-09-03T10:20:25.014Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHDFDRAR2XD243R8VYK6C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:55.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:25.011" grpc.time_ms=2.599 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2309gitlab # [ 442.906044] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.518Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"217b4c4f2cf493480a064b44","created_at":"2026-09-03T10:20:24.736Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.738Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-217b4c4f2cf493480a064b44: done: 0.056202 sec","job_status":"done","queue_duration_s":0.724609,"scheduling_latency_s":0.724646,"redis_calls":7,"redis_duration_s":0.003077,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002289,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056202,"completed_at":"2026-09-03T10:20:25.518Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2310gitlab # [ 442.947740] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.521Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"be79162377e41146a700fb46","created_at":"2026-09-03T10:20:24.738Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.741Z","job_size_bytes":19,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-be79162377e41146a700fb46: start","job_status":"start","queue_duration_s":0.780459,"scheduling_latency_s":0.780489}2311gitlab # [ 442.963495] gitaly[958]: time="2026-09-03T10:20:25.243Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:20:55.041" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:25.241" grpc.time_ms=1.721 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2312gitlab # [ 442.976081] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.565Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"be79162377e41146a700fb46","created_at":"2026-09-03T10:20:24.738Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.741Z","job_size_bytes":19,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-be79162377e41146a700fb46: done: 0.04357 sec","job_status":"done","queue_duration_s":0.780459,"scheduling_latency_s":0.780489,"redis_calls":8,"redis_duration_s":0.00299,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00229,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0007,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04357,"completed_at":"2026-09-03T10:20:25.565Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2313gitlab # [ 443.017188] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.567Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"35e1da6b46f92da812de7fab","created_at":"2026-09-03T10:20:24.743Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.745Z","job_size_bytes":66,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-35e1da6b46f92da812de7fab: start","job_status":"start","queue_duration_s":0.822714,"scheduling_latency_s":0.822754}2314gitlab # [ 443.033848] gitaly[958]: time="2026-09-03T10:20:25.295Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHCY5K65ZQZ1QP4BP8727 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:55.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:25.293" grpc.time_ms=1.263 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2315gitlab # [ 443.046118] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.604Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"35e1da6b46f92da812de7fab","created_at":"2026-09-03T10:20:24.743Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.745Z","job_size_bytes":66,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-35e1da6b46f92da812de7fab: done: 0.036631 sec","job_status":"done","queue_duration_s":0.822714,"scheduling_latency_s":0.822754,"redis_calls":7,"redis_duration_s":0.002407,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000669,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036631,"completed_at":"2026-09-03T10:20:25.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2316gitlab # [ 443.088924] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.606Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"dc1e92872cd9e1d4d8628300","created_at":"2026-09-03T10:20:24.745Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.747Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-dc1e92872cd9e1d4d8628300: start","job_status":"start","queue_duration_s":0.859227,"scheduling_latency_s":0.85925}2317gitlab # [ 443.104758] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.641Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"dc1e92872cd9e1d4d8628300","created_at":"2026-09-03T10:20:24.745Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.747Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-dc1e92872cd9e1d4d8628300: done: 0.035625 sec","job_status":"done","queue_duration_s":0.859227,"scheduling_latency_s":0.85925,"redis_calls":7,"redis_duration_s":0.002143,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000646,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035625,"completed_at":"2026-09-03T10:20:25.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2318gitlab # [ 443.146070] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.644Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d258d55206eda50f042278f5","created_at":"2026-09-03T10:20:24.747Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.749Z","job_size_bytes":6,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d258d55206eda50f042278f5: start","job_status":"start","queue_duration_s":0.894871,"scheduling_latency_s":0.894892}2319gitlab # [ 443.161667] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.662Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d258d55206eda50f042278f5","created_at":"2026-09-03T10:20:24.747Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.749Z","job_size_bytes":6,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d258d55206eda50f042278f5: done: 0.018527 sec","job_status":"done","queue_duration_s":0.894871,"scheduling_latency_s":0.894892,"redis_calls":7,"redis_duration_s":0.001887,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018527,"completed_at":"2026-09-03T10:20:25.662Z","load_balancing_strategy":"primary","db_duration_s":0.000663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2320gitlab # [ 443.202852] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.664Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e2af8858f08074d9db6ec4c9","created_at":"2026-09-03T10:20:24.749Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.752Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e2af8858f08074d9db6ec4c9: start","job_status":"start","queue_duration_s":0.911989,"scheduling_latency_s":0.912008}2321gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2322gitlab # [ 443.217927] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.693Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e2af8858f08074d9db6ec4c9","created_at":"2026-09-03T10:20:24.749Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.752Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e2af8858f08074d9db6ec4c9: done: 0.029141 sec","job_status":"done","queue_duration_s":0.911989,"scheduling_latency_s":0.912008,"redis_calls":7,"redis_duration_s":0.002017,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001425,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029141,"completed_at":"2026-09-03T10:20:25.693Z","load_balancing_strategy":"primary","db_duration_s":0.005571,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2323gitlab # [ 443.256719] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.694Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d8ac80a60abf23171dba6665","created_at":"2026-09-03T10:20:24.808Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.810Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d8ac80a60abf23171dba6665: start","job_status":"start","queue_duration_s":0.88415,"scheduling_latency_s":0.884167}2324gitlab # [ 443.271701] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.726Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d8ac80a60abf23171dba6665","created_at":"2026-09-03T10:20:24.808Z","trace_propagation_headers":{"sentry-trace":"2d806edd10ca46828e6e350f42a48a00-9b3131e8d5654ee1","baggage":"sentry-trace_id=2d806edd10ca46828e6e350f42a48a00,sentry-release=Unknown"},"correlation_id":"01M1KCHBGMTZ4GCM0SWB690G1G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:24.810Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d8ac80a60abf23171dba6665: done: 0.031177 sec","job_status":"done","queue_duration_s":0.88415,"scheduling_latency_s":0.884167,"redis_calls":8,"redis_duration_s":0.002156,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001677,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031177,"completed_at":"2026-09-03T10:20:25.726Z","load_balancing_strategy":"primary","db_duration_s":0.006571,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2325gitlab # [ 443.312901] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.727Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"63ae224425fe679f65c8c817","created_at":"2026-09-03T10:20:25.219Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-5faeb36224d146e8","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:25.222Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-63ae224425fe679f65c8c817: start","job_status":"start","queue_duration_s":0.504595,"scheduling_latency_s":0.504612}2326gitlab # [ 443.328791] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:25.765Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"63ae224425fe679f65c8c817","created_at":"2026-09-03T10:20:25.219Z","trace_propagation_headers":{"sentry-trace":"b3d066e82d884377b1c59b30f91e5af6-5faeb36224d146e8","baggage":"sentry-trace_id=b3d066e82d884377b1c59b30f91e5af6,sentry-release=Unknown"},"correlation_id":"01M1KCHCY5K65ZQZ1QP4BP8727","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:25.222Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-63ae224425fe679f65c8c817: done: 0.037655 sec","job_status":"done","queue_duration_s":0.504595,"scheduling_latency_s":0.504612,"redis_calls":7,"redis_duration_s":0.0018,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001308,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000492,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037655,"completed_at":"2026-09-03T10:20:25.765Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007783,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2327gitlab # [ 443.441746] gitaly[958]: time="2026-09-03T10:20:26.290Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHEPH5JMF3KVJCBKSMV28 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:56.288" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:26.289" grpc.time_ms=1.095 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2328gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.29 seconds)2329gitlab # [ 443.507158] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1612330gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2331gitlab # [ 444.674841] gitaly[958]: time="2026-09-03T10:20:27.523Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHFWQPRHVSEDBD8VD6Z7E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:57.521" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:27.521" grpc.time_ms=1.337 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2332gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.21 seconds)2333gitlab # [ 444.722384] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1542334gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2335gitlab # [ 445.886841] gitaly[958]: time="2026-09-03T10:20:28.735Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHH2JAPFCCKFCZFPTCXV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:58.733" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:28.734" grpc.time_ms=1.269 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2336gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.22 seconds)2337gitlab # [ 445.936857] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1592338gitlab # [ 446.417500] nix-daemon[3902]: accepted connection from pid 4597, user nixbot2339gitlab # [ 446.424185] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:29.271Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1f3ba373dc4d9652cc8f503c","created_at":"2026-09-03T10:20:29.250Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0b09df557db9258953e7de6e0ed9318","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.267Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1f3ba373dc4d9652cc8f503c: start","job_status":"start","queue_duration_s":0.003316,"scheduling_latency_s":0.00335}2340gitlab # [ 446.466362] nix-daemon[3902]: accepted connection from pid 4567, user nixbot2341gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2342gitlab # [ 447.097572] gitaly[958]: time="2026-09-03T10:20:29.946Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHJ8MTV4T473WE4X3NQE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:20:59.844" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:29.944" grpc.time_ms=1.316 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2343gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.22 seconds)2344gitlab # [ 447.157796] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1622345gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2346gitlab # [ 448.307178] gitaly[958]: time="2026-09-03T10:20:31.156Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHKENCTXQ557E3ZM82Y4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:21:01.053" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:31.154" grpc.time_ms=1.206 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2347gitlab # [ 448.353789] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1402348gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.20 seconds)2349gitlab # [ 448.436482] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1f3ba373dc4d9652cc8f503c","created_at":"2026-09-03T10:20:29.250Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0b09df557db9258953e7de6e0ed9318","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.267Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1f3ba373dc4d9652cc8f503c: done: 2.008794 sec","job_status":"done","queue_duration_s":0.003316,"scheduling_latency_s":0.00335,"redis_calls":564,"redis_duration_s":1.140507,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001641,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001337,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":1.137529,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.964097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":2.008794,"completed_at":"2026-09-03T10:20:31.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2350gitlab # [ 448.527931] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"5a1b7359cf5bfd7c673377e9","created_at":"2026-09-03T10:20:29.278Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"757aec83db746a0f5a778bc51fd94ec4","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.310Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-5a1b7359cf5bfd7c673377e9: start","job_status":"start","queue_duration_s":1.976482,"scheduling_latency_s":1.976511}2351gitlab # [ 448.544840] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"5a1b7359cf5bfd7c673377e9","created_at":"2026-09-03T10:20:29.278Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"757aec83db746a0f5a778bc51fd94ec4","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.310Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-5a1b7359cf5bfd7c673377e9: done: 0.026201 sec","job_status":"done","queue_duration_s":1.976482,"scheduling_latency_s":1.976511,"redis_calls":10,"redis_duration_s":0.003117,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002031,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035188,"duration_s":0.026201,"completed_at":"2026-09-03T10:20:31.313Z","load_balancing_strategy":"primary","db_duration_s":0.000851,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2352gitlab # [ 448.587123] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5397c55dcb13fbbefa071071","created_at":"2026-09-03T10:20:29.342Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73311cd25ebc644c17566f5f8928b5c1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.380Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5397c55dcb13fbbefa071071: start","job_status":"start","queue_duration_s":1.93393,"scheduling_latency_s":1.933948}2353gitlab # [ 448.601201] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.348Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"5397c55dcb13fbbefa071071","created_at":"2026-09-03T10:20:29.342Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73311cd25ebc644c17566f5f8928b5c1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.380Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-5397c55dcb13fbbefa071071: done: 0.034172 sec","job_status":"done","queue_duration_s":1.93393,"scheduling_latency_s":1.933948,"redis_calls":9,"redis_duration_s":0.002576,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034172,"completed_at":"2026-09-03T10:20:31.348Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2354gitlab # [ 448.649581] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.350Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0eae9763221ae0e71c2e8f91","created_at":"2026-09-03T10:20:29.405Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ea7f49e9841a6a47c458c55e7bed402","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.428Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0eae9763221ae0e71c2e8f91: start","job_status":"start","queue_duration_s":1.922101,"scheduling_latency_s":1.922119}2355gitlab # [ 448.664155] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0eae9763221ae0e71c2e8f91","created_at":"2026-09-03T10:20:29.405Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ea7f49e9841a6a47c458c55e7bed402","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.428Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0eae9763221ae0e71c2e8f91: done: 0.090584 sec","job_status":"done","queue_duration_s":1.922101,"scheduling_latency_s":1.922119,"redis_calls":11,"redis_duration_s":0.003715,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001518,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00095686,"exclusive_lock_hold_duration_s":0.04577942,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.090584,"completed_at":"2026-09-03T10:20:31.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040991,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2356gitlab # [ 448.708102] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.443Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1bf6c78ee616356d9531c086","created_at":"2026-09-03T10:20:29.448Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85fca76adaf567b15745b5ae29082188","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.474Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1bf6c78ee616356d9531c086: start","job_status":"start","queue_duration_s":1.969398,"scheduling_latency_s":1.969422}2357gitlab # [ 448.722384] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.459Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2358gitlab # [ 448.725798] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"1bf6c78ee616356d9531c086","created_at":"2026-09-03T10:20:29.448Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85fca76adaf567b15745b5ae29082188","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.474Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-1bf6c78ee616356d9531c086: done: 0.02566 sec","job_status":"done","queue_duration_s":1.969398,"scheduling_latency_s":1.969422,"redis_calls":9,"redis_duration_s":0.0028799999999999997,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000716,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001588,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000576,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02566,"completed_at":"2026-09-03T10:20:31.469Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2359gitlab # [ 448.767757] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.471Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"672736f6def94adc151def7c","created_at":"2026-09-03T10:20:29.512Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8359d55a97af7b04a435c5d4f084c670","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.535Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-672736f6def94adc151def7c: start","job_status":"start","queue_duration_s":1.9359,"scheduling_latency_s":1.935919}2360gitlab # [ 448.780946] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"672736f6def94adc151def7c","created_at":"2026-09-03T10:20:29.512Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8359d55a97af7b04a435c5d4f084c670","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.535Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-672736f6def94adc151def7c: done: 0.051772 sec","job_status":"done","queue_duration_s":1.9359,"scheduling_latency_s":1.935919,"redis_calls":10,"redis_duration_s":0.003192,"redis_read_bytes":9,"redis_write_bytes":1039,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00066,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000688,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056486,"duration_s":0.051772,"completed_at":"2026-09-03T10:20:31.522Z","load_balancing_strategy":"primary","db_duration_s":0.020672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2361gitlab # [ 448.822526] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b4983bed39c1d68b6e037891","created_at":"2026-09-03T10:20:29.559Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01896788feb214f07df371a9c4288603","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.582Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b4983bed39c1d68b6e037891: start","job_status":"start","queue_duration_s":1.942605,"scheduling_latency_s":1.942626}2362gitlab # [ 448.836878] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.557Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"b4983bed39c1d68b6e037891","created_at":"2026-09-03T10:20:29.559Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01896788feb214f07df371a9c4288603","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.582Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-b4983bed39c1d68b6e037891: done: 0.033267 sec","job_status":"done","queue_duration_s":1.942605,"scheduling_latency_s":1.942626,"redis_calls":11,"redis_duration_s":0.003578,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000629,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001502,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001447,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070738,"exclusive_lock_hold_duration_s":0.00741498,"duration_s":0.033267,"completed_at":"2026-09-03T10:20:31.557Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2363gitlab # [ 448.878499] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"df4990c9ce3470a934a68863","created_at":"2026-09-03T10:20:29.623Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9fed171604a042d43890bccd5dfac4a","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.643Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-df4990c9ce3470a934a68863: start","job_status":"start","queue_duration_s":1.916222,"scheduling_latency_s":1.916243}2364gitlab # [ 448.891784] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"df4990c9ce3470a934a68863","created_at":"2026-09-03T10:20:29.623Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9fed171604a042d43890bccd5dfac4a","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.643Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-df4990c9ce3470a934a68863: done: 0.034148 sec","job_status":"done","queue_duration_s":1.916222,"scheduling_latency_s":1.916243,"redis_calls":10,"redis_duration_s":0.003078,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000562,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000668,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005346,"duration_s":0.034148,"completed_at":"2026-09-03T10:20:31.594Z","load_balancing_strategy":"primary","db_duration_s":0.00689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2365gitlab # [ 448.932471] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"86eea5f4054878bbae894eba","created_at":"2026-09-03T10:20:29.669Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d1e24700f38e871b7ce68d626f68b28","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.690Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-86eea5f4054878bbae894eba: start","job_status":"start","queue_duration_s":1.906127,"scheduling_latency_s":1.906148}2366gitlab # [ 448.946167] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.618Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"86eea5f4054878bbae894eba","created_at":"2026-09-03T10:20:29.669Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d1e24700f38e871b7ce68d626f68b28","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.690Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-86eea5f4054878bbae894eba: done: 0.022507 sec","job_status":"done","queue_duration_s":1.906127,"scheduling_latency_s":1.906148,"redis_calls":11,"redis_duration_s":0.002885,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000598,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000277,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034638,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.022507,"completed_at":"2026-09-03T10:20:31.618Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2367gitlab # [ 448.989332] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.620Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e9c726323c85ae0565677fe4","created_at":"2026-09-03T10:20:29.719Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078347802dc5d68738dc2ef9d821e2ea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.742Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e9c726323c85ae0565677fe4: start","job_status":"start","queue_duration_s":1.877915,"scheduling_latency_s":1.877932}2368gitlab # [ 449.124279] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.973Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e9c726323c85ae0565677fe4","created_at":"2026-09-03T10:20:29.719Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078347802dc5d68738dc2ef9d821e2ea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.742Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e9c726323c85ae0565677fe4: done: 0.35314 sec","job_status":"done","queue_duration_s":1.877915,"scheduling_latency_s":1.877932,"redis_calls":13,"redis_duration_s":0.0039299999999999995,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001078,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001416,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.277088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060866,"exclusive_lock_hold_duration_s":0.32487076,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.35314,"completed_at":"2026-09-03T10:20:31.973Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.089237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2369gitlab # [ 449.212439] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2220a461d7d34c12c1e55a6c","created_at":"2026-09-03T10:20:29.768Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ae6fe9ea702fcc2b73af398a34892b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:20:29.789Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2220a461d7d34c12c1e55a6c: start","job_status":"start","queue_duration_s":2.185909,"scheduling_latency_s":2.185936}2370gitlab # [ 449.230439] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:31.992Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2371gitlab # [ 449.234708] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2220a461d7d34c12c1e55a6c","created_at":"2026-09-03T10:20:29.768Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ae6fe9ea702fcc2b73af398a34892b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-03T10:20:29.789Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2220a461d7d34c12c1e55a6c: done: 0.025463 sec","job_status":"done","queue_duration_s":2.185909,"scheduling_latency_s":2.185936,"redis_calls":9,"redis_duration_s":0.002582,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000607,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000542,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025463,"completed_at":"2026-09-03T10:20:32.001Z","load_balancing_strategy":"primary","db_duration_s":0.000802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2372gitlab # [ 449.283543] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.003Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"512039b0075f30788a556e13","created_at":"2026-09-03T10:20:29.810Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a702dccb82c968a18ee2bebf1f2a192","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.840Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-512039b0075f30788a556e13: start","job_status":"start","queue_duration_s":2.162395,"scheduling_latency_s":2.162418}2373gitlab # [ 449.297197] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.038Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"512039b0075f30788a556e13","created_at":"2026-09-03T10:20:29.810Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a702dccb82c968a18ee2bebf1f2a192","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.840Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-512039b0075f30788a556e13: done: 0.035704 sec","job_status":"done","queue_duration_s":2.162395,"scheduling_latency_s":2.162418,"redis_calls":10,"redis_duration_s":0.002636,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001534,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000615,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042536,"duration_s":0.035704,"completed_at":"2026-09-03T10:20:32.038Z","load_balancing_strategy":"primary","db_duration_s":0.007248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2374gitlab # [ 449.338475] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.040Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d20baa9480ffe85aef1f048c","created_at":"2026-09-03T10:20:29.862Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec0261e35c526ed6906cdab7b7143538","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:20:29.884Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d20baa9480ffe85aef1f048c: start","job_status":"start","queue_duration_s":2.156316,"scheduling_latency_s":2.156334}2375gitlab # [ 449.351660] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.054Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2376gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2377gitlab # [ 449.355466] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d20baa9480ffe85aef1f048c","created_at":"2026-09-03T10:20:29.862Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec0261e35c526ed6906cdab7b7143538","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-03T10:20:29.884Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d20baa9480ffe85aef1f048c: done: 0.021848 sec","job_status":"done","queue_duration_s":2.156316,"scheduling_latency_s":2.156334,"redis_calls":9,"redis_duration_s":0.002334,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000503,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021848,"completed_at":"2026-09-03T10:20:32.062Z","load_balancing_strategy":"primary","db_duration_s":0.000779,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2378gitlab # [ 449.394522] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryCheck::DispatchWorker","jid":"63124bd3c41e6377362ffb64","created_at":"2026-09-03T10:20:29.905Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83a3211a6d25c6fbd8d22c059cf56450","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:bce0277309dfde25386eb0f6b49415f8ec269d6e1b0147ad102f2a38c86bd6e2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.928Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::DispatchWorker JID-63124bd3c41e6377362ffb64: start","job_status":"start","queue_duration_s":2.135609,"scheduling_latency_s":2.135627}2379gitlab # [ 449.407791] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryCheck::DispatchWorker","jid":"63124bd3c41e6377362ffb64","created_at":"2026-09-03T10:20:29.905Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83a3211a6d25c6fbd8d22c059cf56450","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:bce0277309dfde25386eb0f6b49415f8ec269d6e1b0147ad102f2a38c86bd6e2","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.928Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::DispatchWorker JID-63124bd3c41e6377362ffb64: done: 0.056685 sec","job_status":"done","queue_duration_s":2.135609,"scheduling_latency_s":2.135627,"gitaly_calls":1,"gitaly_duration_s":0.003821,"redis_calls":22,"redis_duration_s":0.008699,"redis_read_bytes":41,"redis_write_bytes":2937,"redis_cache_calls":5,"redis_cache_duration_s":0.000922,"redis_cache_read_bytes":23,"redis_cache_write_bytes":95,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001049,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":658,"redis_queues_calls":2,"redis_queues_duration_s":0.000563,"redis_queues_read_bytes":3,"redis_queues_write_bytes":917,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002161,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":758,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004004,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":509,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.020430140000000003,"duration_s":0.056685,"completed_at":"2026-09-03T10:20:32.120Z","load_balancing_strategy":"primary","db_duration_s":0.001807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2380gitlab # [ 449.453790] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.124Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b67ac6cd7a24c84a732b708a","created_at":"2026-09-03T10:20:29.951Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.981Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b67ac6cd7a24c84a732b708a: start","job_status":"start","queue_duration_s":2.142722,"scheduling_latency_s":2.142761}2381gitlab # [ 449.467299] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.199Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b67ac6cd7a24c84a732b708a","created_at":"2026-09-03T10:20:29.951Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:29.981Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b67ac6cd7a24c84a732b708a: done: 0.075606 sec","job_status":"done","queue_duration_s":2.142722,"scheduling_latency_s":2.142761,"redis_calls":18,"redis_duration_s":0.006697,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001656,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000477,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003369,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075606,"completed_at":"2026-09-03T10:20:32.199Z","load_balancing_strategy":"primary","db_duration_s":0.004383,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2382gitlab # [ 449.509978] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"755d9b3d981f1e076d9bf81b","created_at":"2026-09-03T10:20:30.016Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccd1efe961cc98599e250a8d43b3ad14","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.039Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-755d9b3d981f1e076d9bf81b: start","job_status":"start","queue_duration_s":2.162121,"scheduling_latency_s":2.162148}2383gitlab # [ 449.524172] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.341Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2384gitlab # [ 449.527257] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"755d9b3d981f1e076d9bf81b","created_at":"2026-09-03T10:20:30.016Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccd1efe961cc98599e250a8d43b3ad14","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.039Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-755d9b3d981f1e076d9bf81b: done: 0.147444 sec","job_status":"done","queue_duration_s":2.162121,"scheduling_latency_s":2.162148,"redis_calls":9,"redis_duration_s":0.0027400000000000002,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000706,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.136374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.147444,"completed_at":"2026-09-03T10:20:32.349Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2385gitlab # [ 449.569814] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.350Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d38a1f71cf9c0af2d87a0642","created_at":"2026-09-03T10:20:30.070Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf91215306ba53ccdf7bff3afa865877","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.090Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d38a1f71cf9c0af2d87a0642: start","job_status":"start","queue_duration_s":2.260072,"scheduling_latency_s":2.260096}2386gitlab # [ 449.584335] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.381Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d38a1f71cf9c0af2d87a0642","created_at":"2026-09-03T10:20:30.070Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf91215306ba53ccdf7bff3afa865877","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.090Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d38a1f71cf9c0af2d87a0642: done: 0.030329 sec","job_status":"done","queue_duration_s":2.260072,"scheduling_latency_s":2.260096,"redis_calls":9,"redis_duration_s":0.001369,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030329,"completed_at":"2026-09-03T10:20:32.381Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2387gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.27 seconds)2388gitlab # [ 449.626395] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.383Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"654fe50f4bbf7a07f7bac4eb","created_at":"2026-09-03T10:20:30.115Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf00b96a8b5c6acbe6078abfe2721b3d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.138Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-654fe50f4bbf7a07f7bac4eb: start","job_status":"start","queue_duration_s":2.244938,"scheduling_latency_s":2.244958}2389gitlab # [ 449.639635] gitaly[958]: time="2026-09-03T10:20:32.390Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHMN782AXVGKSK2D0B607 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:21:02.288" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:32.389" grpc.time_ms=1.166 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2390gitlab # [ 449.653386] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1682391gitlab # [ 449.655950] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"654fe50f4bbf7a07f7bac4eb","created_at":"2026-09-03T10:20:30.115Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf00b96a8b5c6acbe6078abfe2721b3d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.138Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-654fe50f4bbf7a07f7bac4eb: done: 0.058594 sec","job_status":"done","queue_duration_s":2.244938,"scheduling_latency_s":2.244958,"redis_calls":10,"redis_duration_s":0.001582,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000332,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003255,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.00335644},"duration_s":0.058594,"completed_at":"2026-09-03T10:20:32.441Z","load_balancing_strategy":"primary","db_duration_s":0.023592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2392gitlab # [ 449.698073] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.443Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"92b133c14d2ac5e022b28c86","created_at":"2026-09-03T10:20:30.172Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd0f9a7be8cdd150c6e52a273483d99e","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.192Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-92b133c14d2ac5e022b28c86: start","job_status":"start","queue_duration_s":2.250224,"scheduling_latency_s":2.250243}2393gitlab # [ 449.712268] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"92b133c14d2ac5e022b28c86","created_at":"2026-09-03T10:20:30.172Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd0f9a7be8cdd150c6e52a273483d99e","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.192Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-92b133c14d2ac5e022b28c86: done: 0.034424 sec","job_status":"done","queue_duration_s":2.250224,"scheduling_latency_s":2.250243,"redis_calls":13,"redis_duration_s":0.002049,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000325,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000818,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030236,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.034424,"completed_at":"2026-09-03T10:20:32.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2394gitlab # [ 449.756044] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e99327d749214ad90905a6b0","created_at":"2026-09-03T10:20:30.249Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.267Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e99327d749214ad90905a6b0: start","job_status":"start","queue_duration_s":2.211559,"scheduling_latency_s":2.211578}2395gitlab # [ 449.769327] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.598Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e99327d749214ad90905a6b0","created_at":"2026-09-03T10:20:30.249Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.267Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e99327d749214ad90905a6b0: done: 0.119603 sec","job_status":"done","queue_duration_s":2.211559,"scheduling_latency_s":2.211578,"redis_calls":70,"redis_duration_s":0.017579,"redis_read_bytes":118,"redis_write_bytes":42480,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002582,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001031,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013345,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000621,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.119603,"completed_at":"2026-09-03T10:20:32.598Z","load_balancing_strategy":"primary","db_duration_s":0.002855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2396gitlab # [ 449.811155] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.600Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"40eca199ba7f5772e6d348ab","created_at":"2026-09-03T10:20:30.286Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9460f59cfee21760761792b371973bd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.307Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-40eca199ba7f5772e6d348ab: start","job_status":"start","queue_duration_s":2.292895,"scheduling_latency_s":2.292911}2397gitlab # [ 449.825466] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.613Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2398gitlab # [ 449.829163] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"40eca199ba7f5772e6d348ab","created_at":"2026-09-03T10:20:30.286Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9460f59cfee21760761792b371973bd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.307Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-40eca199ba7f5772e6d348ab: done: 0.021713 sec","job_status":"done","queue_duration_s":2.292895,"scheduling_latency_s":2.292911,"redis_calls":9,"redis_duration_s":0.002791,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021713,"completed_at":"2026-09-03T10:20:32.622Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2399gitlab # [ 449.871442] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::CleanupPackageRegistryWorker","jid":"26747e558b46420bbae428ca","created_at":"2026-09-03T10:20:30.330Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1553814f1599ce2fbeb197da25b2c15","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31163b2078644913f89feb2245c5fa62ac9118d29e04cf14ce0b5def57bba8ea","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.351Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::CleanupPackageRegistryWorker JID-26747e558b46420bbae428ca: start","job_status":"start","queue_duration_s":2.271947,"scheduling_latency_s":2.271963}2400gitlab # [ 449.885367] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::CleanupPackageRegistryWorker","jid":"26747e558b46420bbae428ca","created_at":"2026-09-03T10:20:30.330Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1553814f1599ce2fbeb197da25b2c15","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31163b2078644913f89feb2245c5fa62ac9118d29e04cf14ce0b5def57bba8ea","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.351Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::CleanupPackageRegistryWorker JID-26747e558b46420bbae428ca: done: 0.105847 sec","job_status":"done","queue_duration_s":2.271947,"scheduling_latency_s":2.271963,"redis_calls":10,"redis_duration_s":0.0029620000000000002,"redis_read_bytes":9,"redis_write_bytes":1125,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":10,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068162,"extra.packages_cleanup_package_registry_worker.pending_destruction_package_files_count":0,"extra.packages_cleanup_package_registry_worker.processing_package_files_count":0,"extra.packages_cleanup_package_registry_worker.error_package_files_count":0,"extra.packages_cleanup_package_registry_worker.pending_cleanup_policies_count":0,"duration_s":0.105847,"completed_at":"2026-09-03T10:20:32.729Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036067,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2401gitlab # [ 449.930213] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.732Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"78fbe4713e804bb9f27fc1ec","created_at":"2026-09-03T10:20:30.375Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"6115ea9087d67070f8235365cd9972e6","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.396Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-78fbe4713e804bb9f27fc1ec: start","job_status":"start","queue_duration_s":2.336094,"scheduling_latency_s":2.336132}2402gitlab # [ 449.943898] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.776Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"78fbe4713e804bb9f27fc1ec","created_at":"2026-09-03T10:20:30.375Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"6115ea9087d67070f8235365cd9972e6","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.396Z","job_size_bytes":3,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-78fbe4713e804bb9f27fc1ec: done: 0.043868 sec","job_status":"done","queue_duration_s":2.336094,"scheduling_latency_s":2.336132,"redis_calls":10,"redis_duration_s":0.003545,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000752,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056006,"duration_s":0.043868,"completed_at":"2026-09-03T10:20:32.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000936,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2403gitlab # [ 449.984752] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"61d546fa622dac49312459eb","created_at":"2026-09-03T10:20:30.415Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec12f94b29c3ad26a37ce06944cc65cd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.436Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-61d546fa622dac49312459eb: start","job_status":"start","queue_duration_s":2.342802,"scheduling_latency_s":2.34283}2404gitlab # [ 449.999048] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.829Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"61d546fa622dac49312459eb","created_at":"2026-09-03T10:20:30.415Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec12f94b29c3ad26a37ce06944cc65cd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.436Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-61d546fa622dac49312459eb: done: 0.05094 sec","job_status":"done","queue_duration_s":2.342802,"scheduling_latency_s":2.34283,"redis_calls":20,"redis_duration_s":0.005395,"redis_read_bytes":23,"redis_write_bytes":4315,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001651,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000813,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2020,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002319,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000612,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040922,"duration_s":0.05094,"completed_at":"2026-09-03T10:20:32.829Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2405gitlab # [ 450.041398] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.831Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"448584e6dc59ad3c2f2ac034","created_at":"2026-09-03T10:20:30.455Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"175c27ea111e93e5deda10fa4cda0504","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.476Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-448584e6dc59ad3c2f2ac034: start","job_status":"start","queue_duration_s":2.355705,"scheduling_latency_s":2.355726}2406gitlab # [ 450.054457] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"448584e6dc59ad3c2f2ac034","created_at":"2026-09-03T10:20:30.455Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"175c27ea111e93e5deda10fa4cda0504","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.476Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-448584e6dc59ad3c2f2ac034: done: 0.033806 sec","job_status":"done","queue_duration_s":2.355705,"scheduling_latency_s":2.355726,"redis_calls":11,"redis_duration_s":0.002993,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000501,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001202,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00066718,"exclusive_lock_hold_duration_s":0.00885284,"duration_s":0.033806,"completed_at":"2026-09-03T10:20:32.865Z","load_balancing_strategy":"primary","db_duration_s":0.005349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2407gitlab # [ 450.095771] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"13346306e41687c2a42b6bbc","created_at":"2026-09-03T10:20:30.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d680dfbdf0aaad9fcdf33fe73ae13b09","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.523Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-13346306e41687c2a42b6bbc: start","job_status":"start","queue_duration_s":2.344141,"scheduling_latency_s":2.34416}2408gitlab # [ 450.110012] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.937Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"13346306e41687c2a42b6bbc","created_at":"2026-09-03T10:20:30.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d680dfbdf0aaad9fcdf33fe73ae13b09","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.523Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-13346306e41687c2a42b6bbc: done: 0.069781 sec","job_status":"done","queue_duration_s":2.344141,"scheduling_latency_s":2.34416,"redis_calls":13,"redis_duration_s":0.003994,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001029,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001487,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001478,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006247,"exclusive_lock_hold_duration_s":0.03964494,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.069781,"completed_at":"2026-09-03T10:20:32.937Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025057,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2409gitlab # [ 450.153313] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"e586eeece32d364ecd417d6e","created_at":"2026-09-03T10:20:30.545Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbe04c8bf0796424684dca26cb759989","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.567Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-e586eeece32d364ecd417d6e: start","job_status":"start","queue_duration_s":2.371228,"scheduling_latency_s":2.371248}2410gitlab # [ 450.166118] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"e586eeece32d364ecd417d6e","created_at":"2026-09-03T10:20:30.545Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbe04c8bf0796424684dca26cb759989","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.567Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-e586eeece32d364ecd417d6e: done: 0.027464 sec","job_status":"done","queue_duration_s":2.371228,"scheduling_latency_s":2.371248,"redis_calls":10,"redis_duration_s":0.003021,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000486,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000565,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036202,"duration_s":0.027464,"completed_at":"2026-09-03T10:20:32.966Z","load_balancing_strategy":"primary","db_duration_s":0.000762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2411gitlab # [ 450.206316] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"350c7fe57e8fd8456c6b956a","created_at":"2026-09-03T10:20:30.596Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf018119822e6734e4bb8715baf39a5d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.617Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-350c7fe57e8fd8456c6b956a: start","job_status":"start","queue_duration_s":2.350675,"scheduling_latency_s":2.350691}2412gitlab # [ 450.220827] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:32.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"350c7fe57e8fd8456c6b956a","created_at":"2026-09-03T10:20:30.596Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf018119822e6734e4bb8715baf39a5d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.617Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-350c7fe57e8fd8456c6b956a: done: 0.030403 sec","job_status":"done","queue_duration_s":2.350675,"scheduling_latency_s":2.350691,"redis_calls":9,"redis_duration_s":0.002269,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001291,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030403,"completed_at":"2026-09-03T10:20:32.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2413gitlab # [ 450.262224] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7d88e1118e739af7f320d195","created_at":"2026-09-03T10:20:30.645Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b76071b800cb9f758d74e09f65223c8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.672Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7d88e1118e739af7f320d195: start","job_status":"start","queue_duration_s":2.327133,"scheduling_latency_s":2.32715}2414gitlab # [ 450.277375] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7d88e1118e739af7f320d195","created_at":"2026-09-03T10:20:30.645Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b76071b800cb9f758d74e09f65223c8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.672Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7d88e1118e739af7f320d195: done: 0.032666 sec","job_status":"done","queue_duration_s":2.327133,"scheduling_latency_s":2.32715,"redis_calls":12,"redis_duration_s":0.003128,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001528,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00944156,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.032666,"completed_at":"2026-09-03T10:20:33.032Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004235,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2415gitlab # [ 450.320873] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"467b0c9ac8be09c5bff2ae6a","created_at":"2026-09-03T10:20:30.697Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a37e7964c2dae814cded2091c7c876b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.719Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-467b0c9ac8be09c5bff2ae6a: start","job_status":"start","queue_duration_s":2.314463,"scheduling_latency_s":2.314478}2416gitlab # [ 450.335050] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.045Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2417gitlab # [ 450.338701] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"467b0c9ac8be09c5bff2ae6a","created_at":"2026-09-03T10:20:30.697Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a37e7964c2dae814cded2091c7c876b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.719Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-467b0c9ac8be09c5bff2ae6a: done: 0.01866 sec","job_status":"done","queue_duration_s":2.314463,"scheduling_latency_s":2.314478,"redis_calls":9,"redis_duration_s":0.0022069999999999998,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01866,"completed_at":"2026-09-03T10:20:33.052Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2418gitlab # [ 450.381133] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.054Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"807c7d23d665fcd1cdb36d25","created_at":"2026-09-03T10:20:30.749Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7547e36c95b8b6d300fc394a1f139901","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.769Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-807c7d23d665fcd1cdb36d25: start","job_status":"start","queue_duration_s":2.284265,"scheduling_latency_s":2.28428}2419gitlab # [ 450.394284] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.072Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"807c7d23d665fcd1cdb36d25","created_at":"2026-09-03T10:20:30.749Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7547e36c95b8b6d300fc394a1f139901","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.769Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-807c7d23d665fcd1cdb36d25: done: 0.018227 sec","job_status":"done","queue_duration_s":2.284265,"scheduling_latency_s":2.28428,"redis_calls":11,"redis_duration_s":0.002694,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000456,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018227,"completed_at":"2026-09-03T10:20:33.072Z","load_balancing_strategy":"primary","db_duration_s":0.00063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2420gitlab # [ 450.435879] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"14c71518bae496eb58eddca4","created_at":"2026-09-03T10:20:30.791Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb646a7927d4c65930c01d327bd1a8d9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.811Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-14c71518bae496eb58eddca4: start","job_status":"start","queue_duration_s":2.262834,"scheduling_latency_s":2.26285}2421gitlab # [ 450.450194] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"14c71518bae496eb58eddca4","created_at":"2026-09-03T10:20:30.791Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb646a7927d4c65930c01d327bd1a8d9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.811Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-14c71518bae496eb58eddca4: done: 0.02214 sec","job_status":"done","queue_duration_s":2.262834,"scheduling_latency_s":2.26285,"redis_calls":11,"redis_duration_s":0.002868,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055786,"exclusive_lock_hold_duration_s":0.00335542,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.02214,"completed_at":"2026-09-03T10:20:33.095Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2422gitlab # [ 450.493905] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"82140c759534296aa194c84b","created_at":"2026-09-03T10:20:30.833Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6928c328c6fca75125e518b7e06ea4af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:20:30.853Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-82140c759534296aa194c84b: start","job_status":"start","queue_duration_s":2.243668,"scheduling_latency_s":2.243683}2423gitlab # [ 450.506733] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"82140c759534296aa194c84b","created_at":"2026-09-03T10:20:30.833Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6928c328c6fca75125e518b7e06ea4af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-03T10:20:30.853Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-82140c759534296aa194c84b: done: 0.032048 sec","job_status":"done","queue_duration_s":2.243668,"scheduling_latency_s":2.243683,"redis_calls":9,"redis_duration_s":0.002431,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000517,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000547,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032048,"completed_at":"2026-09-03T10:20:33.129Z","load_balancing_strategy":"primary","db_duration_s":0.005862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2424gitlab # [ 450.546911] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"572b0d26d4fb3cddff2d07c8","created_at":"2026-09-03T10:20:30.877Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1eab791c19e75f721b3147cf75392ef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.899Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-572b0d26d4fb3cddff2d07c8: start","job_status":"start","queue_duration_s":2.231937,"scheduling_latency_s":2.231951}2425gitlab # [ 450.561548] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.153Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"572b0d26d4fb3cddff2d07c8","created_at":"2026-09-03T10:20:30.877Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1eab791c19e75f721b3147cf75392ef","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.899Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-572b0d26d4fb3cddff2d07c8: done: 0.02208 sec","job_status":"done","queue_duration_s":2.231937,"scheduling_latency_s":2.231951,"redis_calls":11,"redis_duration_s":0.002918,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001098,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000513,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02208,"completed_at":"2026-09-03T10:20:33.153Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2426gitlab # [ 450.603345] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"980a98f53b4f1ab341bc5d3f","created_at":"2026-09-03T10:20:30.957Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"600cd30a6e13b166db216ee0b3a3824b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.975Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-980a98f53b4f1ab341bc5d3f: start","job_status":"start","queue_duration_s":2.180135,"scheduling_latency_s":2.18015}2427gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2428gitlab # [ 450.617264] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.196Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"980a98f53b4f1ab341bc5d3f","created_at":"2026-09-03T10:20:30.957Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"600cd30a6e13b166db216ee0b3a3824b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:30.975Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-980a98f53b4f1ab341bc5d3f: done: 0.041158 sec","job_status":"done","queue_duration_s":2.180135,"scheduling_latency_s":2.18015,"redis_calls":9,"redis_duration_s":0.002393,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.041158,"completed_at":"2026-09-03T10:20:33.196Z","load_balancing_strategy":"primary","db_duration_s":0.019633,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2429gitlab # [ 450.657578] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"09e0def7a8a04a06a7d2c195","created_at":"2026-09-03T10:20:30.994Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a847b4d4d0570aad3b6b00cdae8a009","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:31.015Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-09e0def7a8a04a06a7d2c195: start","job_status":"start","queue_duration_s":2.183157,"scheduling_latency_s":2.183173}2430gitlab # [ 450.670927] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"09e0def7a8a04a06a7d2c195","created_at":"2026-09-03T10:20:30.994Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-c318e04f1d834fd3","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a847b4d4d0570aad3b6b00cdae8a009","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:31.015Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-09e0def7a8a04a06a7d2c195: done: 0.018815 sec","job_status":"done","queue_duration_s":2.183157,"scheduling_latency_s":2.183173,"redis_calls":9,"redis_duration_s":0.002558,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001225,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018815,"completed_at":"2026-09-03T10:20:33.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2431gitlab # [ 450.713600] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"repository_check","args":["default"],"class":"RepositoryCheck::BatchWorker","jid":"de5c78e80966f3fecf331d22","created_at":"2026-09-03T10:20:32.091Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-031e67bf4c2b4e5e","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"RepositoryCheck::DispatchWorker","correlation_id":"83a3211a6d25c6fbd8d22c059cf56450","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b22e99a0acd71e0cf2962b3ba955d6f34621628300baf74d4c861b087bc17605","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.102Z","job_size_bytes":11,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::BatchWorker JID-de5c78e80966f3fecf331d22: start","job_status":"start","queue_duration_s":1.116402,"scheduling_latency_s":1.116416}2432gitlab # [ 450.727194] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"repository_check","args":["default"],"class":"RepositoryCheck::BatchWorker","jid":"de5c78e80966f3fecf331d22","created_at":"2026-09-03T10:20:32.091Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-031e67bf4c2b4e5e","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"RepositoryCheck::DispatchWorker","correlation_id":"83a3211a6d25c6fbd8d22c059cf56450","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b22e99a0acd71e0cf2962b3ba955d6f34621628300baf74d4c861b087bc17605","size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.102Z","job_size_bytes":11,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::BatchWorker JID-de5c78e80966f3fecf331d22: done: 0.0294 sec","job_status":"done","queue_duration_s":1.116402,"scheduling_latency_s":1.116416,"redis_calls":13,"redis_duration_s":0.0033020000000000002,"redis_read_bytes":12,"redis_write_bytes":1387,"redis_cache_calls":1,"redis_cache_duration_s":0.000287,"redis_cache_read_bytes":1,"redis_cache_write_bytes":37,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0005,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001155,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":513,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006751999999999999,"duration_s":0.0294,"completed_at":"2026-09-03T10:20:33.248Z","load_balancing_strategy":"primary","db_duration_s":0.003565,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2433gitlab # [ 450.769764] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.249Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a103650aa47cf64c8371f9d3","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a103650aa47cf64c8371f9d3: start","job_status":"start","queue_duration_s":1.062218,"scheduling_latency_s":1.062233}2434gitlab # [ 450.781948] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.264Z","message":"No existing merge request to be cleaned up."}2435gitlab # [ 450.783563] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.278Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a103650aa47cf64c8371f9d3","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a103650aa47cf64c8371f9d3: done: 0.028975 sec","job_status":"done","queue_duration_s":1.062218,"scheduling_latency_s":1.062233,"redis_calls":12,"redis_duration_s":0.003032,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028975,"completed_at":"2026-09-03T10:20:33.278Z","load_balancing_strategy":"primary","db_duration_s":0.005258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2436gitlab # [ 450.824631] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.280Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c7effdb50f79abb69411bebf","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c7effdb50f79abb69411bebf: start","job_status":"start","queue_duration_s":1.092933,"scheduling_latency_s":1.092949}2437gitlab # [ 450.836760] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.291Z","message":"No existing merge request to be cleaned up."}2438gitlab # [ 450.838408] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.299Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c7effdb50f79abb69411bebf","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c7effdb50f79abb69411bebf: done: 0.018996 sec","job_status":"done","queue_duration_s":1.092933,"scheduling_latency_s":1.092949,"redis_calls":10,"redis_duration_s":0.002273,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018996,"completed_at":"2026-09-03T10:20:33.299Z","load_balancing_strategy":"primary","db_duration_s":0.002371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2439gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.25 seconds)2440gitlab # [ 450.876651] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.301Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"78bbb08692d640e975dbe071","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-78bbb08692d640e975dbe071: start","job_status":"start","queue_duration_s":1.113715,"scheduling_latency_s":1.113732}2441gitlab # [ 450.888522] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.311Z","message":"No existing merge request to be cleaned up."}2442gitlab # [ 450.890514] gitaly[958]: time="2026-09-03T10:20:33.667Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHNXTZ9RV7GHGPAG0PAFD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:21:03.566" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:33.666" grpc.time_ms=0.845 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2443gitlab # [ 450.904515] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1102444gitlab # [ 450.907105] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.335Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"78bbb08692d640e975dbe071","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-78bbb08692d640e975dbe071: done: 0.034354 sec","job_status":"done","queue_duration_s":1.113715,"scheduling_latency_s":1.113732,"redis_calls":10,"redis_duration_s":0.004824,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002811,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034354,"completed_at":"2026-09-03T10:20:33.335Z","load_balancing_strategy":"primary","db_duration_s":0.002716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2445gitlab # [ 450.945185] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.337Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"895c2c0d4f8de8cbdf561d57","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-895c2c0d4f8de8cbdf561d57: start","job_status":"start","queue_duration_s":1.149537,"scheduling_latency_s":1.149783}2446gitlab # [ 450.957298] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.348Z","message":"No existing merge request to be cleaned up."}2447gitlab # [ 450.959138] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.357Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"895c2c0d4f8de8cbdf561d57","created_at":"2026-09-03T10:20:32.159Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-3ce82d43723b4e19","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aa4138325fc5cdf93eadc62b47764236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.187Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-895c2c0d4f8de8cbdf561d57: done: 0.020897 sec","job_status":"done","queue_duration_s":1.149537,"scheduling_latency_s":1.149783,"redis_calls":10,"redis_duration_s":0.0026639999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020897,"completed_at":"2026-09-03T10:20:33.357Z","load_balancing_strategy":"primary","db_duration_s":0.002699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2448gitlab # [ 450.996693] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4d05aeaf8490673539789a6","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4d05aeaf8490673539789a6: start","job_status":"start","queue_duration_s":0.771384,"scheduling_latency_s":0.7714}2449gitlab # [ 451.008478] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4d05aeaf8490673539789a6","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4d05aeaf8490673539789a6: done: 0.033343 sec","job_status":"done","queue_duration_s":0.771384,"scheduling_latency_s":0.7714,"redis_calls":15,"redis_duration_s":0.004255,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000627,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001366,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002262,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033343,"completed_at":"2026-09-03T10:20:33.392Z","load_balancing_strategy":"primary","db_duration_s":0.001065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2450gitlab # [ 451.048583] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0122118e5c01c138c7a011d8","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0122118e5c01c138c7a011d8: start","job_status":"start","queue_duration_s":0.808341,"scheduling_latency_s":0.808381}2451gitlab # [ 451.060702] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0122118e5c01c138c7a011d8","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0122118e5c01c138c7a011d8: done: 0.039961 sec","job_status":"done","queue_duration_s":0.808341,"scheduling_latency_s":0.808381,"redis_calls":13,"redis_duration_s":0.004873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039961,"completed_at":"2026-09-03T10:20:33.435Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2452gitlab # [ 451.099783] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccdf201c12842dbc03a4d19a","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccdf201c12842dbc03a4d19a: start","job_status":"start","queue_duration_s":0.850963,"scheduling_latency_s":0.850996}2453gitlab # [ 451.112139] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccdf201c12842dbc03a4d19a","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccdf201c12842dbc03a4d19a: done: 0.029281 sec","job_status":"done","queue_duration_s":0.850963,"scheduling_latency_s":0.850996,"redis_calls":13,"redis_duration_s":0.003771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002492,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029281,"completed_at":"2026-09-03T10:20:33.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2454gitlab # [ 451.150764] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a215e93a9d30281a04678f07","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a215e93a9d30281a04678f07: start","job_status":"start","queue_duration_s":0.882642,"scheduling_latency_s":0.882665}2455gitlab # [ 451.163201] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a215e93a9d30281a04678f07","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a215e93a9d30281a04678f07: done: 0.032165 sec","job_status":"done","queue_duration_s":0.882642,"scheduling_latency_s":0.882665,"redis_calls":13,"redis_duration_s":0.004298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00283,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032165,"completed_at":"2026-09-03T10:20:33.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2456gitlab # [ 451.202283] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08cc50d9abb95a09009831ee","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08cc50d9abb95a09009831ee: start","job_status":"start","queue_duration_s":0.918844,"scheduling_latency_s":0.918886}2457gitlab # [ 451.214828] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08cc50d9abb95a09009831ee","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08cc50d9abb95a09009831ee: done: 0.045125 sec","job_status":"done","queue_duration_s":0.918844,"scheduling_latency_s":0.918886,"redis_calls":13,"redis_duration_s":0.005971,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045125,"completed_at":"2026-09-03T10:20:33.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2458gitlab # [ 451.254407] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5c03b1be21000a07c10609","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5c03b1be21000a07c10609: start","job_status":"start","queue_duration_s":0.96869,"scheduling_latency_s":0.96873}2459gitlab # [ 451.266941] nix-daemon[3902]: accepted connection from pid 4567, user nixbot2460gitlab # [ 451.267940] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e5c03b1be21000a07c10609","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e5c03b1be21000a07c10609: done: 0.048413 sec","job_status":"done","queue_duration_s":0.96869,"scheduling_latency_s":0.96873,"redis_calls":13,"redis_duration_s":0.007716000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004137,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.048413,"completed_at":"2026-09-03T10:20:33.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2461gitlab # [ 451.306880] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ce45b5e67bbfddfd22d55af","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ce45b5e67bbfddfd22d55af: start","job_status":"start","queue_duration_s":1.020814,"scheduling_latency_s":1.020853}2462gitlab # [ 451.320165] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ce45b5e67bbfddfd22d55af","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ce45b5e67bbfddfd22d55af: done: 0.03805 sec","job_status":"done","queue_duration_s":1.020814,"scheduling_latency_s":1.020853,"redis_calls":13,"redis_duration_s":0.003892,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001304,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002588,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03805,"completed_at":"2026-09-03T10:20:33.646Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2463gitlab # [ 451.359210] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12bb7e1b221662c5e09fca2f","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12bb7e1b221662c5e09fca2f: start","job_status":"start","queue_duration_s":1.061253,"scheduling_latency_s":1.06128}2464gitlab # [ 451.371582] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12bb7e1b221662c5e09fca2f","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12bb7e1b221662c5e09fca2f: done: 0.023498 sec","job_status":"done","queue_duration_s":1.061253,"scheduling_latency_s":1.06128,"redis_calls":13,"redis_duration_s":0.002297,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001462,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023498,"completed_at":"2026-09-03T10:20:33.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2465gitlab # [ 451.410882] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5becd7be8f4cd76c04b6706","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5becd7be8f4cd76c04b6706: start","job_status":"start","queue_duration_s":1.086144,"scheduling_latency_s":1.086163}2466gitlab # [ 451.423135] gitaly[958]: time="2026-09-03T10:20:34.189Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHPDJR217M5N3K32F9HDJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:21:04.087" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:34.188" grpc.time_ms=0.891 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2467gitlab # [ 451.435993] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5becd7be8f4cd76c04b6706","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5becd7be8f4cd76c04b6706: done: 0.019639 sec","job_status":"done","queue_duration_s":1.086144,"scheduling_latency_s":1.086163,"redis_calls":13,"redis_duration_s":0.002001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019639,"completed_at":"2026-09-03T10:20:33.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2468gitlab # [ 451.475106] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15094274994eea34464b92ee","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15094274994eea34464b92ee: start","job_status":"start","queue_duration_s":1.107178,"scheduling_latency_s":1.107201}2469gitlab # [ 451.487439] gitaly[958]: time="2026-09-03T10:20:34.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHPDJR217M5N3K32F9HDJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:44.149" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:34.229" grpc.time_ms=16.917 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2470gitlab # [ 451.501469] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15094274994eea34464b92ee","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15094274994eea34464b92ee: done: 0.018114 sec","job_status":"done","queue_duration_s":1.107178,"scheduling_latency_s":1.107201,"redis_calls":13,"redis_duration_s":0.001897,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001289,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018114,"completed_at":"2026-09-03T10:20:33.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2471gitlab # [ 451.541477] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"038c6ad572b9361958e529c2","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-038c6ad572b9361958e529c2: start","job_status":"start","queue_duration_s":1.126361,"scheduling_latency_s":1.126376}2472gitlab # [ 451.554143] nixbot[3674]: {"ts": "2026-09-03T10:20:34.344Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 \"HTTP/1.1 201 Created\""}2473gitlab # [ 451.556972] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2422474gitlab # [ 451.559443] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"038c6ad572b9361958e529c2","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-038c6ad572b9361958e529c2: done: 0.017289 sec","job_status":"done","queue_duration_s":1.126361,"scheduling_latency_s":1.126376,"redis_calls":13,"redis_duration_s":0.002437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001456,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017289,"completed_at":"2026-09-03T10:20:33.731Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2475gitlab # [ 451.598480] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"519c4a9a7ad1f7a84e3c35e9","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-519c4a9a7ad1f7a84e3c35e9: start","job_status":"start","queue_duration_s":1.144846,"scheduling_latency_s":1.144862}2476gitlab # [ 451.610978] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"519c4a9a7ad1f7a84e3c35e9","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-519c4a9a7ad1f7a84e3c35e9: done: 0.017219 sec","job_status":"done","queue_duration_s":1.144846,"scheduling_latency_s":1.144862,"redis_calls":13,"redis_duration_s":0.0017490000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017219,"completed_at":"2026-09-03T10:20:33.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2477gitlab # [ 451.649996] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4a7610fab6125d354d58f7a","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4a7610fab6125d354d58f7a: start","job_status":"start","queue_duration_s":1.163201,"scheduling_latency_s":1.163218}2478gitlab # [ 451.662404] gitaly[958]: time="2026-09-03T10:20:34.448Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHPNH27J21NDPNMBMFBEK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:21:04.346" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:34.447" grpc.time_ms=1.127 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2479gitlab # [ 451.675214] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4a7610fab6125d354d58f7a","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4a7610fab6125d354d58f7a: done: 0.018028 sec","job_status":"done","queue_duration_s":1.163201,"scheduling_latency_s":1.163218,"redis_calls":13,"redis_duration_s":0.003315,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018028,"completed_at":"2026-09-03T10:20:33.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2480gitlab # [ 451.713994] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"567cec585b63eaeda2f50faa","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-567cec585b63eaeda2f50faa: start","job_status":"start","queue_duration_s":1.183047,"scheduling_latency_s":1.183064}2481gitlab # [ 451.726569] gitaly[958]: time="2026-09-03T10:20:34.517Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHPNH27J21NDPNMBMFBEK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:44.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:34.504" grpc.time_ms=13.399 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2482gitlab # [ 451.739790] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"567cec585b63eaeda2f50faa","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-567cec585b63eaeda2f50faa: done: 0.017683 sec","job_status":"done","queue_duration_s":1.183047,"scheduling_latency_s":1.183064,"redis_calls":13,"redis_duration_s":0.003064,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017683,"completed_at":"2026-09-03T10:20:33.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2483gitlab # [ 451.779804] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90fcca73f79e89be719c42ce","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90fcca73f79e89be719c42ce: start","job_status":"start","queue_duration_s":1.202603,"scheduling_latency_s":1.20262}2484gitlab # [ 451.792047] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2352485gitlab # [ 451.794441] nixbot[3674]: {"ts": "2026-09-03T10:20:34.591Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 \"HTTP/1.1 201 Created\""}2486gitlab # [ 451.797140] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90fcca73f79e89be719c42ce","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90fcca73f79e89be719c42ce: done: 0.016232 sec","job_status":"done","queue_duration_s":1.202603,"scheduling_latency_s":1.20262,"redis_calls":13,"redis_duration_s":0.002852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016232,"completed_at":"2026-09-03T10:20:33.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2487gitlab # [ 451.836229] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6a06c85d35872c716d3e15e","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6a06c85d35872c716d3e15e: start","job_status":"start","queue_duration_s":1.220931,"scheduling_latency_s":1.220975}2488gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses2489gitlab # [ 451.848320] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6a06c85d35872c716d3e15e","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6a06c85d35872c716d3e15e: done: 0.038201 sec","job_status":"done","queue_duration_s":1.220931,"scheduling_latency_s":1.220975,"redis_calls":13,"redis_duration_s":0.003694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002299,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038201,"completed_at":"2026-09-03T10:20:33.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2490gitlab # [ 451.886926] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0a89ce696be5272ce37c1c6","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0a89ce696be5272ce37c1c6: start","job_status":"start","queue_duration_s":1.262339,"scheduling_latency_s":1.262378}2491gitlab # [ 451.898712] gitaly[958]: time="2026-09-03T10:20:34.680Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHPXDZ1XJT2ERK1K8G5YN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:21:04.678" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:34.679" grpc.time_ms=0.819 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2492gitlab # [ 451.910405] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0a89ce696be5272ce37c1c6","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0a89ce696be5272ce37c1c6: done: 0.032777 sec","job_status":"done","queue_duration_s":1.262339,"scheduling_latency_s":1.262378,"redis_calls":13,"redis_duration_s":0.0035919999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001187,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002405,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032777,"completed_at":"2026-09-03T10:20:33.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2493gitlab # [ 451.949195] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33c89d3d473d0b9b021e5fcb","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33c89d3d473d0b9b021e5fcb: start","job_status":"start","queue_duration_s":1.297231,"scheduling_latency_s":1.297257}2494gitlab # [ 451.962527] gitaly[958]: time="2026-09-03T10:20:34.742Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHPXDZ1XJT2ERK1K8G5YN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-03T10:20:44.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-03T10:20:34.729" grpc.time_ms=12.786 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2495gitlab # [ 451.975834] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33c89d3d473d0b9b021e5fcb","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33c89d3d473d0b9b021e5fcb: done: 0.024049 sec","job_status":"done","queue_duration_s":1.297231,"scheduling_latency_s":1.297257,"redis_calls":13,"redis_duration_s":0.003517,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002345,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024049,"completed_at":"2026-09-03T10:20:33.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2496gitlab # [ 452.015018] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bd29997dafc77b13598e94a","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bd29997dafc77b13598e94a: start","job_status":"start","queue_duration_s":1.323156,"scheduling_latency_s":1.323177}2497gitlab # [ 452.027630] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2162498gitlab # [ 452.030242] gitaly[958]: time="2026-09-03T10:20:34.831Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=7 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=7 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1KCHPDJR217M5N3K32F9HDJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-03T10:21:04.622" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:34.823" grpc.time_ms=7.626 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2499gitlab # [ 452.042932] nixbot[3674]: {"ts": "2026-09-03T10:20:34.826Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/05c5fe8317a89e596d33e6555010cf0e802e5771 \"HTTP/1.1 201 Created\""}2500gitlab # [ 452.045820] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bd29997dafc77b13598e94a","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bd29997dafc77b13598e94a: done: 0.021898 sec","job_status":"done","queue_duration_s":1.323156,"scheduling_latency_s":1.323177,"redis_calls":13,"redis_duration_s":0.0031969999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021898,"completed_at":"2026-09-03T10:20:33.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2501gitlab # [ 452.084213] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dd9c5775c4d5ada7989b2d5","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dd9c5775c4d5ada7989b2d5: start","job_status":"start","queue_duration_s":1.347061,"scheduling_latency_s":1.347082}2502gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses, in 0.22 seconds)2503(finished: subtest: eval and build post commit statuses, in 10.07 seconds)2504(finished: run the VM test script, in 525.89 seconds)2505gitlab # [ 452.096969] gitaly[958]: time="2026-09-03T10:20:34.882Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1KCHQ3YPFFZH1NVX72N0JPA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-03T10:21:04.780" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-03T10:20:34.881" grpc.time_ms=0.593 pid=958 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2506gitlab # [ 452.109338] gitlab-workhorse[893]: gitlab 2001:db8:1::1 - - [2026/09/03:10:20:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/05c5fe8317a89e596d33e6555010cf0e802e5771/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1172507test script finished in 525.92s2508cleanup2509kill QemuMachine (pid 46)2510gitlab # [ 452.111856] sidekiq-cluster[3622]: {"severity":"INFO","time":"2026-09-03T10:20:33.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dd9c5775c4d5ada7989b2d5","created_at":"2026-09-03T10:20:32.502Z","trace_propagation_headers":{"sentry-trace":"98d1ff4fce0e47139a4544631536933e-d98d291fa5194c07","baggage":"sentry-trace_id=98d1ff4fce0e47139a4544631536933e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fb56fe49bcc811e1928c014494330282","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-03T10:20:32.587Z","job_size_bytes":2,"pid":3622,"sidekiq_tid":"m8um","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dd9c5775c4d5ada7989b2d5: done: 0.019196 sec","job_status":"done","queue_duration_s":1.347061,"scheduling_latency_s":1.347082,"redis_calls":13,"redis_duration_s":0.003126,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cachqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2511(finished: cleanup, in 1.90 seconds)2512additionally exposed symbols:2513 gitlab,2514 vlan1,2515 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2516[]2517[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-03T10:18:48.408Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2518[]2519[]2520[]2521[]2522[]2523[]2524[]2525[]2526[]2527[]2528[]2529[]2530[]2531[]2532[]2533[]2534[]2535[]2536[]2537[]2538[]2539[]2540[]2541[]2542[]2543[]2544[]2545[]2546[]2547[]2548[]2549[]2550[]2551[]2552[]2553[]2554[]2555[]2556[]2557[]2558[]2559[]2560[]2561[]2562[]2563[]2564[]2565[]2566[]2567[]2568[]2569[]2570[]2571[]2572[]2573[]2574[]2575[]2576[]2577[]2578[]2579[]2580[]2581[]2582[]2583[]2584[]2585[]2586[]2587[]2588[]2589[]2590[]2591[]2592[]2593[]2594[]2595[]2596[]2597[]2598[]2599[]2600[]2601[]2602[]2603[]2604[]2605[]2606[]2607[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-03T10:20:24.377940Z', 'started_at': '2026-09-03T10:20:24.735660Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2608[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2609[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2610[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2611[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2612[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2613[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2614[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2615[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2616[{'id': 5, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-03T10:20:24.702Z', 'started_at': None, 'finished_at': '2026-09-03T10:20:34.280Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '05c5fe8317a89e596d33e6555010cf0e802e5771', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-03T10:20:34.555Z', 'started_at': None, 'finished_at': '2026-09-03T10:20:34.770Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]