vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #487
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.A12GeYB4aV', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: ad94c804-d2e5-4a50-9dce-8af212c3e4ef16gitlab # Superblock backups stored on blocks:17(finished: start all VMs, in 66.68 seconds)18gitlab # 32768, 98304, 163840, 22937619gitlab: waiting for unit setup-gitlab.service20gitlab # 21gitlab: waiting for the VM to finish booting22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ywj1nm676jxd139znwh4sl8iy19przzp-nixos-system-gitlab-test/init regInfo=/nix/store/mhhjq1g65k6z3mlhm5lp9x4wg1i89h45-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/mhhjq1g65k6z3mlhm5lp9x4wg1i89h45-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000032] arm-pv: using stolen time PV108gitlab # [ 0.000377] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000604] Console: colour dummy device 80x25110gitlab # [ 0.000612] printk: legacy console [tty0] enabled111gitlab # [ 0.000803] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000810] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000883] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.001020] landlock: Up and running.115gitlab # [ 0.001023] Yama: becoming mindful.116gitlab # [ 0.001442] LSM support for eBPF active117gitlab # [ 0.001666] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001794] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002979] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003755] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003760] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003921] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.005039] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.005132] EFI services will not be available.125gitlab # [ 0.005317] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005848] Detected PIPT I-cache on CPU1127gitlab # [ 0.005907] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005981] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.006053] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.006963] Detected PIPT I-cache on CPU2131gitlab # [ 0.007030] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007120] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007167] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.007967] Detected PIPT I-cache on CPU3135gitlab # [ 0.008034] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008107] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.008178] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.008475] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.008505] SMP: Total of 4 processors activated.140gitlab # [ 0.008508] CPU: All CPU(s) started at EL1141gitlab # [ 0.008516] CPU features: detected: Branch Target Identification142gitlab # [ 0.008520] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.008523] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.008526] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.008529] CPU features: detected: Common not Private translations146gitlab # [ 0.008532] CPU features: detected: CRC32 instructions147gitlab # [ 0.008535] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.008538] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.008541] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.008544] CPU features: detected: E0PD151gitlab # [ 0.008546] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.008549] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.008552] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.008555] CPU features: detected: Fine Grained Traps155gitlab # [ 0.008557] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.008561] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.008564] CPU features: detected: LSE atomic instructions158gitlab # [ 0.008566] CPU features: detected: Privileged Access Never159gitlab # [ 0.008569] CPU features: detected: PMUv3160gitlab # [ 0.008571] CPU features: detected: RAS Extension Support161gitlab # [ 0.008573] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.008575] CPU features: detected: Random Number Generator163gitlab # [ 0.008578] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.008580] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.008583] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.008586] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.008674] alternatives: applying system-wide alternatives168gitlab # [ 0.011581] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.011841] Memory: 5963804K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.013230] devtmpfs: initialized171gitlab # [ 0.016494] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.016568] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.016739] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.016744] 0 pages in range for non-PLT usage175gitlab # [ 0.016746] 508288 pages in range for PLT usage176gitlab # [ 0.016874] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.017643] DMI not present or invalid.178gitlab # [ 0.021078] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.023577] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.023841] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.026147] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.026168] audit: initializing netlink subsys (disabled)183gitlab # [ 0.026392] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.026872] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.026875] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.026880] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.026883] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.026885] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.026939] cpuidle: using governor ladder190gitlab # [ 0.026959] cpuidle: using governor menu191gitlab # [ 0.027166] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.027296] ASID allocator initialised with 65536 entries193gitlab # [ 0.028446] Serial: AMBA PL011 UART driver194gitlab # [ 0.033608] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.033835] printk: console [ttyAMA0] enabled196gitlab # [ 0.039880] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.039890] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.039894] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.039899] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.039903] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.039907] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.039911] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.039915] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.047529] fbcon: Taking over console205gitlab # [ 0.047549] ACPI: Interpreter disabled.206gitlab # [ 0.049238] iommu: Default domain type: Translated207gitlab # [ 0.049247] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.060988] SCSI subsystem initialized209gitlab # [ 0.061618] usbcore: registered new interface driver usbfs210gitlab # [ 0.061653] usbcore: registered new interface driver hub211gitlab # [ 0.061675] usbcore: registered new device driver usb212gitlab # [ 0.062153] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.062159] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.062171] PTP clock support registered215gitlab # [ 0.062226] EDAC MC: Ver: 3.0.0216gitlab # [ 0.062617] scmi_core: SCMI protocol bus registered217gitlab # [ 0.063276] FPGA manager framework218gitlab # [ 0.064530] vgaarb: loaded219gitlab # [ 0.065872] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.068185] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.068220] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.068581] netfs: FS-Cache loaded223gitlab # [ 0.068698] pnp: PnP ACPI: disabled224gitlab # [ 0.073043] NET: Registered PF_INET protocol family225gitlab # [ 0.073999] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.101819] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.101865] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.102007] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.102296] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.102841] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.102923] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.102971] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.103136] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.103283] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.103309] NET: Registered PF_XDP protocol family236gitlab # [ 0.103322] PCI: CLS 0 bytes, default 64237gitlab # [ 0.103536] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.114721] kvm [1]: HYP mode not available239gitlab # [ 0.157476] Initialise system trusted keyrings240gitlab # [ 0.157644] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.158164] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.158256] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.170202] Key type asymmetric registered244gitlab # [ 0.170207] Asymmetric key parser 'x509' registered245gitlab # [ 0.170307] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.170418] io scheduler mq-deadline registered247gitlab # [ 0.170422] io scheduler kyber registered248gitlab # [ 0.178736] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.182330] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.182684] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.182697] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.182706] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.182712] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.182735] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.182753] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.182826] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.182833] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.182837] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.182840] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.182843] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.182906] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.183406] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.183626] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.183639] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.183668] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.183680] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.184193] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.184401] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.184418] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.184451] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.184966] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.185176] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.185188] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.185216] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.185738] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.185967] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.185979] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.186007] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.186553] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.186770] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.186782] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.186810] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.187371] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.187596] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.187608] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.187641] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.188182] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.188400] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.188412] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.188439] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.188451] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.189031] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.189262] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.189290] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.189842] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.190105] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.190133] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.190659] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.190880] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.191168] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.191490] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.191507] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.191536] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.192090] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.192305] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.192317] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.192344] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.192991] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.192997] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.193001] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.193051] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.193097] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.193148] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.193198] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.193247] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.193292] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.193342] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.193393] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.193443] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.193509] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.193559] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.193581] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.193604] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.193627] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.193650] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.193673] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.193697] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.193720] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.193743] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.193766] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.193786] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.193805] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.193823] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.193841] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.193859] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.193889] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.193907] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.193925] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.193943] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.193961] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.193985] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.194008] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.194013] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.194016] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.195437] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.207429] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.209821] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.211993] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.214425] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.216674] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.221350] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.223627] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.225805] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.228026] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.230015] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.234863] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.240556] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.242332] msm_serial: driver initialized359gitlab # [ 0.242487] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.242535] STM32 USART driver initialized361gitlab # [ 0.262176] loop: module loaded362gitlab # [ 0.262360] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.264157] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.266028] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.268121] virtio_blk virtio5: [vdb] 12395568 512-byte logical blocks (6.35 GB/5.91 GiB)366gitlab # [ 0.270277] megasas: 07.734.00.00-rc1367gitlab # [ 0.270934] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.272978] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.272994] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.275104] Using buffer write method371gitlab # [ 0.275132] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.277124] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.277138] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.278818] Using buffer write method375gitlab # [ 0.278832] Concatenating MTD devices:376gitlab # [ 0.278834] (0): "0.flash"377gitlab # [ 0.278836] (1): "0.flash"378gitlab # [ 0.278838] into device "0.flash"379gitlab # [ 0.505474] Freeing initrd memory: 27184K380gitlab # [ 0.511961] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.515378] thunder_xcv, ver 1.0382gitlab # [ 0.515424] thunder_bgx, ver 1.0383gitlab # [ 0.515445] nicpf, ver 1.0384gitlab # [ 0.516051] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.516056] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.516080] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.516085] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.516113] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.516117] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.516138] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.516142] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.516282] sky2: driver version 1.30393gitlab # [ 0.517907] usbcore: registered new interface driver usb-storage394gitlab # [ 0.518098] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.518113] usbserial: USB Serial support registered for generic396gitlab # [ 0.518595] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.518647] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.518737] hv_vmbus: registering driver hyperv_keyboard399gitlab # [ 0.519048] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000400gitlab # [ 0.519813] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.519842] rtc-pl031 9010000.pl031: setting system clock to 2026-11-02T13:10:09 UTC (1793625009)402gitlab # [ 0.520257] i2c_dev: i2c /dev entries driver403gitlab # [ 0.523083] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.523089] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.523351] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.523719] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.525397] hid: raw HID events driver (C) Jiri Kosina408gitlab # [ 0.525677] usbcore: registered new interface driver usbhid409gitlab # [ 0.525681] usbhid: USB HID core driver410gitlab # [ 0.545938] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00411gitlab # [ 0.546764] hub 1-0:1.0: USB hub found412gitlab # [ 0.546814] hub 1-0:1.0: 6 ports detected413gitlab # [ 0.555486] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.557046] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.557224] NET: Registered PF_INET6 protocol family416gitlab # [ 0.558032] Segment Routing with IPv6417gitlab # [ 0.558045] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.558071] NET: Registered PF_PACKET protocol family419gitlab # [ 0.558210] 9pnet: Installing 9P2000 support420gitlab # [ 0.559857] Key type dns_resolver registered421gitlab # [ 0.565711] registered taskstats version 1422gitlab # [ 0.565891] Loading compiled-in X.509 certificates423gitlab # [ 0.573064] Demotion targets for Node 0: null424gitlab # [ 0.573223] Key type .fscrypt registered425gitlab # [ 0.573227] Key type fscrypt-provisioning registered426gitlab # [ 0.573336] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.573353] ima: Allocated hash algorithm: sha1428gitlab # [ 0.573372] ima: No architecture policies found429gitlab # [ 0.574055] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.591081] clk: Disabling unused clocks431gitlab # [ 0.591102] PM: genpd: Disabling unused power domains432gitlab # [ 0.594295] Freeing unused kernel memory: 4736K433gitlab # [ 0.594467] Run /init as init process434gitlab # [ 0.621774] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.793916] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 0.943855] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 0.955807] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 0.955847] systemd[1]: Detected virtualization qemu.439gitlab # [ 0.955885] systemd[1]: Detected architecture arm64.440gitlab # [ 0.955892] systemd[1]: Running in initrd.441gitlab # [ 0.956352] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 0.956495] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.050314] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.135127] systemd[1]: bpf-restrict-fs: LSM BPF program attached445gitlab # [ 1.173948] usb 1-2: new high-speed USB device number 3 using ehci-pci446gitlab # [ 1.199589] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.212772] systemd[1]: Created slice Slice /system/modprobe.448gitlab # [ 1.213022] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.449gitlab # [ 1.213063] systemd[1]: Expecting device /dev/disk/by-label/nix-store...450gitlab # [ 1.213078] systemd[1]: Expecting device /dev/disk/by-label/nixos...451gitlab # [ 1.213102] systemd[1]: Reached target Path Units.452gitlab # [ 1.213120] systemd[1]: Reached target Slice Units.453gitlab # [ 1.213140] systemd[1]: Reached target Swaps.454gitlab # [ 1.213159] systemd[1]: Reached target Timer Units.455gitlab # [ 1.213327] systemd[1]: Listening on D-Bus System Message Bus Socket.456gitlab # [ 1.213500] systemd[1]: Listening on Journal Socket (/dev/log).457gitlab # [ 1.213666] systemd[1]: Listening on Journal Sockets.458gitlab # [ 1.213861] systemd[1]: Listening on udev Control Socket.459gitlab # [ 1.214014] systemd[1]: Listening on udev Kernel Socket.460gitlab # [ 1.214036] systemd[1]: Reached target Socket Units.461gitlab # [ 1.215739] systemd[1]: Starting Create List of Static Device Nodes...462gitlab # [ 1.217368] systemd[1]: Starting Load Kernel Module 9pnet_virtio...463gitlab # [ 1.217446] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs464gitlab # [ 1.219237] systemd[1]: Mounting Kernel Configuration File System...465gitlab # [ 1.235737] systemd[1]: Starting Journal Service...466gitlab # [ 1.237784] systemd[1]: Starting Load Kernel Modules...467gitlab # [ 1.237966] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os468gitlab # [ 1.240027] systemd[1]: Starting Coldplug All udev Devices...469gitlab # [ 1.244742] systemd[1]: Finished Create List of Static Device Nodes.470gitlab # [ 1.247676] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.471gitlab # [ 1.248199] systemd[1]: Finished Load Kernel Module 9pnet_virtio.472gitlab # [ 1.248554] systemd[1]: Mounted Kernel Configuration File System.473gitlab # [ 1.258048] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...474gitlab # [ 1.269455] systemd-journald[95]: Collecting audit messages is disabled.475gitlab # [ 1.271268] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.476gitlab # [ 1.276406] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0477gitlab # [ 1.276590] [drm] features: -virgl +edid -resource_blob -host_visible478gitlab # [ 1.276604] [drm] features: -context_init479gitlab # [ 1.276685] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev480gitlab # [ 1.281780] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.481gitlab # [ 1.282526] [drm] number of scanouts: 1482gitlab # [ 1.282545] [drm] number of cap sets: 0483gitlab # [ 1.284062] systemd[1]: Starting Create Static Device Nodes in /dev...484gitlab # [ 1.286485] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic485gitlab # [ 1.286504] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0486gitlab # [ 1.297379] Console: switching to colour frame buffer device 160x50487gitlab # [ 1.301727] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device488gitlab # [ 1.320237] systemd[1]: Finished Load Kernel Modules.489gitlab # [ 1.322016] systemd[1]: Finished Create Static Device Nodes in /dev.490gitlab # [ 1.322344] systemd[1]: Reached target Preparation for Local File Systems.491gitlab # [ 1.322643] systemd[1]: Reached target Local File Systems.492gitlab # [ 1.324797] systemd[1]: Starting Apply Kernel Variables...493gitlab # [ 1.331386] systemd[1]: Starting Rule-based Manager for Device Events and Files...494gitlab # [ 1.333754] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2495gitlab # [ 1.334422] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0496gitlab # [ 1.345883] systemd-modules-load[96]: Using 4 probe threads497gitlab # [ 1.347328] systemd-modules-load[96]: Module 'virtio_balloon' is built in[ 1.346319] systemd[1]: Started Journal Service.498gitlab # 499gitlab # [ 1.349698] systemd-modules-load[96]: Module 'virtio_console' is built in500gitlab # [ 1.352007] systemd-modules-load[96]: Module 'virtio_rng' is built in501gitlab # [ 1.353346] systemd-modules-load[96]: Inserted module 'dm_mod'502gitlab # [ 1.354403] systemd-modules-load[96]: Inserted module 'virtio_gpu'503gitlab # [ 1.357604] systemd[1]: Starting Create System Files and Directories...504gitlab # [ 1.364090] systemd[1]: Finished Apply Kernel Variables.505gitlab # [ 1.371070] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.375745] systemd-udevd[108]: Using default interface naming scheme 'v261'.507gitlab # [ 1.389145] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.409523] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.451281] systemd-vconsole-setup[131]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.454126] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 1.872421] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 1.873885] systemd[1]: Reached target System Initialization.513gitlab # [ 1.875190] systemd[1]: Reached target Basic System.514gitlab # [ 1.977622] (udev-worker)[139]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 1.982187] (udev-worker)[139]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 1.994580] (udev-worker)[127]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.023043] systemd[1]: Found device /dev/disk/by-label/nix-store.518gitlab # [ 2.027782] systemd[1]: Found device /dev/disk/by-label/nixos.519gitlab # [ 2.030503] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.031715] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.073169] systemd-fsck[153]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.076950] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.241424] systemd[1]: Mounting /sysroot...524gitlab # [ 2.271988] EXT4-fs (vda): mounted filesystem ad94c804-d2e5-4a50-9dce-8af212c3e4ef r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.272448] systemd[1]: Mounted /sysroot.526gitlab # [ 2.273353] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.275954] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.280179] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.281417] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.293267] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.296908] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.299053] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.325373] systemd-sysroot-fstab-check[167]: /sysroot should be mounted in the initrd, will request daemon-reload.534gitlab # [ 2.331308] systemd[1]: Mounted /sysroot/nix/.rw-store.535gitlab # [ 2.332409] systemd[1]: Mounted /sysroot/run.536gitlab # [ 2.341207] systemd[1]: Starting rw-sysroot-nix-store.service...537gitlab # [ 2.343388] systemd[1]: Reload requested from client PID 167 ('systemd-sysroot') (unit initrd-parse-etc.service)...538gitlab # [ 2.345180] systemd[1]: Reloading...539gitlab # [ 2.396869] erofs (device vdb): mounted with root inode @ nid 128.540gitlab # [ 2.486188] systemd[1]: Reloading finished in 142 ms.541gitlab # [ 2.525464] (udev-worker)[123]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory542gitlab # [ 2.529382] (udev-worker)[123]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory543gitlab # [ 2.547932] systemd-sysroot-fstab-check[167]: Requesting initrd-fs.target/start/replace...544gitlab # [ 2.550634] systemd[1]: Mounted /sysroot/nix/.ro-store.545gitlab # [ 2.551427] systemd[1]: Mounted /sysroot/tmp/shared.546gitlab # [ 2.552258] systemd[1]: Mounted /sysroot/tmp/xchg.547gitlab # [ 2.552974] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.548gitlab # [ 2.554001] systemd[1]: Finished rw-sysroot-nix-store.service.549gitlab # [ 2.558882] systemd-sysroot-fstab-check[167]: Requesting swap.target/start/replace...550gitlab # [ 2.563868] systemd[1]: initrd-parse-etc.service: Deactivated successfully.551gitlab # [ 2.565366] systemd[1]: Finished Mountpoints Configured in the Real Root.552gitlab # [ 2.566556] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.553gitlab # [ 2.574923] systemd[1]: Starting rw-sysroot-nix-store.service...554gitlab # [ 2.576206] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.555gitlab # [ 2.577631] systemd[1]: Stopped Virtual Console Setup.556gitlab # [ 2.578680] systemd[1]: Stopping Virtual Console Setup...557gitlab # [ 2.579759] systemd[1]: Starting Virtual Console Setup...558gitlab # [ 2.618278] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.559gitlab # [ 2.620001] systemd[1]: Finished rw-sysroot-nix-store.service.560gitlab # [ 2.634464] systemd-vconsole-setup[199]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.637248] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 3.243766] systemd[1]: Mounting /sysroot/nix/store...563gitlab # [ 3.293904] systemd[1]: Mounted /sysroot/nix/store.564gitlab # [ 3.295152] systemd[1]: Reached target Initrd File Systems.565gitlab # [ 3.296476] systemd[1]: Starting Find NixOS closure...566gitlab # [ 3.297674] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...567gitlab # [ 3.325497] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.568gitlab # [ 3.327433] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.569gitlab # [ 3.336358] systemd[1]: Finished Find NixOS closure.570gitlab # [ 3.337616] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 3.338942] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 3.370773] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 3.372225] systemd[1]: Stopped target Basic System.574gitlab # [ 3.373376] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 3.374611] systemd[1]: Stopped target Path Units.576gitlab # [ 3.375724] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 3.377578] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 3.379402] systemd[1]: Stopped target Slice Units.579gitlab # [ 3.380603] systemd[1]: Stopped target Socket Units.580gitlab # [ 3.381829] systemd[1]: Stopped target System Initialization.581gitlab # [ 3.384256] systemd[1]: Stopped target Swaps.582gitlab # [ 3.385342] systemd[1]: Stopped target Timer Units.583gitlab # [ 3.387496] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 3.389864] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 3.391303] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 3.393088] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 3.394228] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 3.395660] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 3.397491] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 3.399167] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 3.400425] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 3.402009] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 3.403256] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 3.405225] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 3.406928] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 3.408687] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 3.410182] systemd[1]: Stopped target Local File Systems.598gitlab # [ 3.411444] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 3.413048] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 3.414631] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 3.415874] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 3.417645] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 3.419229] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 3.420577] systemd[1]: initrd-cleanup.service: Deactivated successfully.605gitlab # [ 3.422043] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.606gitlab # [ 3.423380] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.607gitlab # [ 3.424922] systemd[1]: Finished Load Kernel Module 9pnet_virtio.608gitlab # [ 3.426157] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.427553] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.428736] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 3.429987] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 3.431314] systemd[1]: systemd-udevd.service: Consumed 1.934s CPU time over 2.080s wall clock time, 32.1M memory peak.613gitlab # [ 3.433284] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 3.434565] systemd[1]: Closed udev Control Socket.615gitlab # [ 3.435467] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 3.436496] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 3.437819] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 3.438880] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 3.440262] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 3.441453] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 3.442657] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 3.466878] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 3.468632] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 3.469893] systemd[1]: Reached target Switch Root.625gitlab # [ 3.471017] systemd[1]: Starting NixOS Activation...626gitlab # [ 3.578828] initrd-nixos-activation-start[227]: booting system configuration /nix/store/ywj1nm676jxd139znwh4sl8iy19przzp-nixos-system-gitlab-test627gitlab # [ 3.612968] initrd-nixos-activation-start[227]: running activation script...628gitlab # [ 3.832363] initrd-nixos-activation-start[251]: setting up /etc...629gitlab # [ 3.932510] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.630gitlab # [ 3.934307] systemd[1]: Finished NixOS Activation.631gitlab # [ 3.935410] systemd[1]: Starting Switch Root...632gitlab # [ 3.971284] systemd[1]: Switching root.633gitlab # [ 4.080536] systemd-journald[95]: Received SIGTERM from PID 1 (systemd).634gitlab # [ 4.629986] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)635gitlab # [ 4.630064] systemd[1]: Detected virtualization qemu.636gitlab # [ 4.630122] systemd[1]: Detected architecture arm64.637gitlab # [ 4.630225] systemd[1]: Detected first boot.638gitlab # [ 4.630984] systemd[1]: Initializing machine ID from random generator.639gitlab # [ 4.815655] systemd[1]: bpf-restrict-fs: LSM BPF program attached640gitlab # [ 4.896753] systemd[1]: Applying preset policy.641gitlab # [ 5.070687] systemd[1]: Populated /etc with preset unit settings.642gitlab # [ 5.216284] systemd[1]: initrd-switch-root.service: Deactivated successfully.643gitlab # [ 5.216658] systemd[1]: Stopped initrd-switch-root.service.644gitlab # [ 5.218531] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.645gitlab # [ 5.222143] systemd[1]: Created slice Slice /system/getty.646gitlab # [ 5.223984] systemd[1]: Created slice GitLab DevOps Platform Slice.647gitlab # [ 5.225910] systemd[1]: Created slice User and Session Slice.648gitlab # [ 5.226474] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.649gitlab # [ 5.226956] systemd[1]: Started Forward Password Requests to Wall Directory Watch.650gitlab # [ 5.227389] systemd[1]: Expecting device /dev/hvc0...651gitlab # [ 5.227845] systemd[1]: Expecting device /dev/ttyAMA0...652gitlab # [ 5.228281] systemd[1]: Reached target Local Encrypted Volumes.653gitlab # [ 5.228708] systemd[1]: Stopped target initrd-fs.target.654gitlab # [ 5.229127] systemd[1]: Stopped target initrd-root-fs.target.655gitlab # [ 5.229549] systemd[1]: Stopped target initrd-switch-root.target.656gitlab # [ 5.230003] systemd[1]: Reached target Virtual Machines and Containers.657gitlab # [ 5.230410] systemd[1]: Reached target Path Units.658gitlab # [ 5.230824] systemd[1]: Reached target Remote File Systems.659gitlab # [ 5.231254] systemd[1]: Reached target Slice Units.660gitlab # [ 5.231677] systemd[1]: Reached target Swaps.661gitlab # [ 5.233081] systemd[1]: Listening on Query the User Interactively for a Password.662gitlab # [ 5.234544] systemd[1]: Listening on Process Core Dump Socket.663gitlab # [ 5.235785] systemd[1]: Listening on Credential Encryption/Decryption.664gitlab # [ 5.237019] systemd[1]: Listening on Factory Reset Management.665gitlab # [ 5.237496] systemd[1]: Listening on Hostname Service Socket.666gitlab # [ 5.243298] systemd[1]: Starting Journal Log Access Socket...667gitlab # [ 5.245119] systemd[1]: Listening on Journal Audit Socket.668gitlab # [ 5.247252] systemd[1]: Listening on Console Output Muting Service Socket.669gitlab # [ 5.248129] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.670gitlab # [ 5.248761] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os671gitlab # [ 5.249320] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki672gitlab # [ 5.252498] systemd[1]: Listening on Disk Repartitioning Service Socket.673gitlab # [ 5.253235] systemd[1]: Listening on udev Control Socket.674gitlab # [ 5.253986] systemd[1]: Listening on udev Varlink Socket.675gitlab # [ 5.258888] systemd[1]: Mounting Huge Pages File System...676gitlab # [ 5.263697] systemd[1]: Mounting POSIX Message Queue File System...677gitlab # [ 5.269456] systemd[1]: Mounting Kernel Debug File System...678gitlab # [ 5.278064] systemd[1]: Mounting Kernel Trace File System...679gitlab # [ 5.282383] systemd[1]: Starting Create List of Static Device Nodes...680gitlab # [ 5.288679] systemd[1]: Starting Load Kernel Module 9pnet_virtio...681gitlab # [ 5.291192] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs682gitlab # [ 5.298062] systemd[1]: Mounting Kernel Configuration File System...683gitlab # [ 5.299985] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm684gitlab # [ 5.303072] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore685gitlab # [ 5.308237] systemd[1]: Starting Load Kernel Module fuse...686gitlab # [ 5.308919] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67687gitlab # [ 5.313677] systemd[1]: Starting Journal Service...688gitlab # [ 5.316557] systemd[1]: Starting Load Kernel Modules...689gitlab # [ 5.321452] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...690gitlab # [ 5.325833] systemd[1]: Starting Remount Root and Kernel File Systems...691gitlab # [ 5.330299] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os692gitlab # [ 5.338097] systemd[1]: Starting Coldplug All udev Devices...693gitlab # [ 5.342869] fuse: init (API version 7.45)694gitlab # [ 5.346186] systemd[1]: Listening on Journal Log Access Socket.695gitlab # [ 5.350216] systemd[1]: Mounted Huge Pages File System.696gitlab # [ 5.350978] systemd-journald[322]: Collecting audit messages is enabled.697gitlab # [ 5.354278] systemd[1]: Mounted POSIX Message Queue File System.698gitlab # [ 5.356178] systemd[1]: Mounted Kernel Debug File System.699gitlab # [ 5.357022] systemd[1]: Mounted Kernel Trace File System.700gitlab # [ 5.358271] systemd[1]: Finished Create List of Static Device Nodes.701gitlab # [ 5.359291] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.702gitlab # [ 5.360192] systemd[1]: Finished Load Kernel Module 9pnet_virtio.703gitlab # [ 5.361017] systemd[1]: Mounted Kernel Configuration File System.704gitlab # [ 5.362076] systemd[1]: modprobe@fuse.service: Deactivated successfully.705gitlab # [ 5.362988] systemd[1]: Finished Load Kernel Module fuse.706gitlab # [ 5.364145] systemd[1]: Finished Load Kernel Modules.707gitlab # [ 5.365945] EXT4-fs (vda): re-mounted ad94c804-d2e5-4a50-9dce-8af212c3e4ef.708gitlab # [ 5.370405] systemd[1]: Mounting FUSE Control File System...709gitlab # [ 5.370977] systemd[1]: Queued start job for default target Multi-User System.710gitlab # [ 5.372446] systemd[1]: Unnecessary job was removed for /dev/vdb.711gitlab # [ 5.373677] systemd[1]: systemd-journald.service: Deactivated successfully.712gitlab # [ 5.375013] systemd-modules-load[323]: Using 3 probe threads713gitlab # [ 5.376113] systemd-modules-load[323]: Module 'atkbd' is built in714gitlab # [ 5.377240] systemd-modules-load[323]: Module 'loop' is built in715gitlab # [ 5.378323] systemd-modules-load[323]: Inserted module 'tls'716gitlab # [ 5.382517] systemd[1]: Starting Firewall...717gitlab # [ 5.394056] systemd[1]: Starting Apply Kernel Variables...718gitlab # [ 5.398303] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...719gitlab # [ 5.407127] systemd[1]: Started Journal Service.720gitlab # [ 5.409121] systemd[1]: Finished Remount Root and Kernel File Systems.721gitlab # [ 5.410543] systemd[1]: Mounted FUSE Control File System.722gitlab # [ 5.416211] systemd-oomd[324]: No swap; memory pressure usage will be degraded723gitlab # [ 5.419225] systemd[1]: Finished Apply Kernel Variables.724gitlab # [ 5.420884] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.725gitlab # [ 5.423316] systemd[1]: Listening on Disk Image Download Service Socket.726gitlab # [ 5.427376] systemd[1]: Starting Flush Journal to Persistent Storage...727gitlab # [ 5.428612] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore728gitlab # [ 5.430556] systemd[1]: Starting Load/Save OS Random Seed...729gitlab # [ 5.431495] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os730gitlab # [ 5.438598] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.731gitlab # [ 5.443116] systemd-journald[322]: Received client request to flush runtime journal.732gitlab # [ 5.470376] systemd[1]: Starting Create Static Device Nodes in /dev...733gitlab # [ 5.473813] systemd[1]: Finished Load/Save OS Random Seed.734gitlab # [ 5.474989] systemd[1]: Reached target First Boot Complete.735gitlab # [ 5.476792] systemd[1]: Finished Create Static Device Nodes in /dev.736gitlab # [ 5.477772] systemd[1]: Reached target Preparation for Local File Systems.737gitlab # [ 5.478941] systemd[1]: Starting Rule-based Manager for Device Events and Files...738gitlab # [ 5.480034] systemd[1]: Finished Flush Journal to Persistent Storage.739gitlab # [ 5.516288] systemd-udevd[367]: Using default interface naming scheme 'v261'.740gitlab # [ 5.547791] systemd[1]: Started Rule-based Manager for Device Events and Files.741gitlab # [ 5.722202] systemd[1]: Finished Firewall.742gitlab # [ 5.984647] systemd[1]: Finished Coldplug All udev Devices.743gitlab # [ 6.044659] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs744gitlab # [ 6.075791] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.745gitlab # [ 6.095245] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.746gitlab # [ 6.119462] (udev-worker)[478]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.747gitlab # [ 6.121964] (udev-worker)[478]: Network interface NamePolicy= disabled on kernel command line.748gitlab # [ 6.144837] systemd[1]: Condition check resulted in Virtio network device being skipped.749gitlab # [ 6.146492] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore750gitlab # [ 6.148626] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67751gitlab # [ 6.151085] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore752gitlab # [ 6.153183] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os753gitlab # [ 6.154475] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os754gitlab # [ 6.159525] (udev-worker)[411]: Network interface NamePolicy= disabled on kernel command line.755gitlab # [ 6.186517] mousedev: PS/2 mouse device common for all mice756gitlab # [ 6.216818] systemd[1]: Mounting /run/wrappers...757gitlab # [ 6.240749] systemd[1]: Mounted /run/wrappers.758gitlab # [ 6.242491] systemd[1]: Reached target Local File Systems.759gitlab # [ 6.243595] systemd[1]: Listening on Boot Loader Control Service Socket.760gitlab # [ 6.245847] systemd[1]: Starting register-nix-paths.service...761gitlab # [ 6.248727] systemd[1]: Starting Create SUID/SGID Wrappers...762gitlab # [ 6.249987] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.763gitlab # [ 6.262620] systemd[1]: Starting Save Transient machine-id to Disk...764gitlab # [ 6.268379] systemd[1]: Starting Create System Files and Directories...765gitlab # [ 6.292102] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.766gitlab # [ 6.297198] systemd[1]: Finished Save Transient machine-id to Disk.767gitlab # [ 6.306514] systemd[1]: Finished Create System Files and Directories.768gitlab # [ 6.309526] systemd[1]: Starting Rebuild Journal Catalog...769gitlab # [ 6.314344] systemd[1]: Starting Record System Boot/Shutdown in UTMP...770gitlab # [ 6.344378] systemd[1]: Finished Record System Boot/Shutdown in UTMP.771gitlab # [ 6.357452] systemd[1]: Finished Rebuild Journal Catalog.772gitlab # [ 6.362808] systemd[1]: Starting Update is Completed...773gitlab # [ 6.379993] systemd[1]: Finished Update is Completed.774gitlab # [ 6.411461] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.775gitlab # [ 6.412861] systemd[1]: Finished Create SUID/SGID Wrappers.776gitlab # [ 6.414085] systemd[1]: suid-sgid-wrappers.service: Consumed 122ms CPU time over 162ms wall clock time, 6.5M memory peak, 1.9M read from disk.777gitlab # [ 6.631572] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3778gitlab # [ 6.669579] systemd[1]: Finished register-nix-paths.service.779gitlab # [ 6.671999] systemd[1]: Reached target System Initialization.780gitlab # [ 6.673031] systemd[1]: Started Discard unused filesystem blocks once a week.781gitlab # [ 6.675639] systemd[1]: Started logrotate.timer.782gitlab # [ 6.678019] systemd[1]: Started Daily Cleanup of Temporary Directories.783gitlab # [ 6.680329] systemd[1]: Reached target Timer Units.784gitlab # [ 6.681187] systemd[1]: Listening on D-Bus System Message Bus Socket.785gitlab # [ 6.682151] systemd[1]: Listening on Nix Daemon Socket.786gitlab # [ 6.698800] systemd[1]: Starting nixbot CI service socket...787gitlab # [ 6.699839] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.788gitlab # [ 6.701410] systemd[1]: Starting D-Bus System Message Bus...789gitlab # [ 6.702259] systemd[1]: Listening on nixbot CI service socket.790gitlab # [ 6.703126] systemd[1]: Reached target Socket Units.791gitlab # [ 6.736681] dbus-broker-launch[568]: Looking up NSS user entry for 'systemd-timesync'...792gitlab # [ 6.738493] dbus-broker-launch[568]: NSS returned no entry for 'systemd-timesync'793gitlab # [ 6.739681] dbus-broker-launch[568]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"794gitlab # [ 6.743606] systemd[1]: Started D-Bus System Message Bus.795gitlab # [ 6.757226] dbus-broker-launch[568]: Ready796gitlab # [ 6.764562] systemd[1]: Reached target Basic System.797gitlab # [ 6.766866] systemd[1]: Started backdoor.service.798gitlab # [ 6.769294] systemd[1]: Starting gitlab-config.service...799gitlab # [ 6.783450] systemd[1]: Starting Import lastlog data into lastlog2 database...800gitlab # [ 6.785160] systemd[1]: Starting Logrotate configuration check...801gitlab # [ 6.789570] systemd[1]: Starting Name Service Cache Daemon (nsncd)...802gitlab # [ 6.795016] systemd[1]: Starting Post-Boot Actions...803gitlab # [ 6.800631] systemd[1]: Started Reset console on configuration changes.804gitlab # connecting to host...805gitlab # [ 6.807504] systemd[1]: Starting resolvconf update...806gitlab # [ 6.808798] logrotate[576]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.807gitlab # [ 6.810994] logrotate[576]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf808gitlab # [ 6.814717] logrotate[576]: note: 'monthly' overrides previously specified 'weekly'809gitlab # [ 6.820379] logrotate[576]: Reading state from file: /var/lib/logrotate.status810gitlab # [ 6.821434] logrotate[576]: state file /var/lib/logrotate.status does not exist811gitlab # [ 6.822531] logrotate[576]: Allocating hash table for state file, size 64 entries812gitlab # [ 6.823540] logrotate[576]: Handling 4 logs813gitlab # [ 6.825555] logrotate[576]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed814gitlab # [ 6.828182] logrotate[576]: considering log /var/log/btmp815gitlab # [ 6.829217] logrotate[576]: Creating new state816gitlab: Guest shell says: b'Spawning backdoor root shell...\n'817gitlab # [ 6.829852] logrotate[576]: Now: 2026-11-02 13:10818gitlab # [ 6.831818] logrotate[576]: Last rotated at 2026-11-02 13:00819gitlab: connected to guest root shell820gitlab # [ 6.835316] logrotate[576]: log does not need rotating (log has already been rotated)821gitlab: (connecting took 7.29 seconds)822gitlab # [ 6.836558] logrotate[576]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed823gitlab: (finished: waiting for the VM to finish booting, in 7.29 seconds)824gitlab # [ 6.838757] logrotate[576]: considering log /var/log/wtmp825gitlab # [ 6.839537] logrotate[576]: Creating new state826gitlab # [ 6.840175] logrotate[576]: Now: 2026-11-02 13:10827gitlab # [ 6.840924] logrotate[576]: Last rotated at 2026-11-02 13:00828gitlab # [ 6.841912] logrotate[576]: log does not need rotating (log has already been rotated)829gitlab # [ 6.843031] logrotate[576]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed830gitlab # [ 6.845082] logrotate[576]: switching euid from 0 to 165 and egid from 0 to 165 (pid 576)831gitlab # [ 6.846386] logrotate[576]: considering log /var/gitlab/state/log/*.log832gitlab # [ 6.847519] logrotate[576]: log /var/gitlab/state/log/*.log does not exist -- skipping833gitlab # [ 6.848741] logrotate[576]: Creating new state834gitlab # [ 6.849390] logrotate[576]: switching euid from 165 to 0 and egid from 165 to 0 (pid 576)835gitlab # [ 6.850622] logrotate[576]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed836gitlab # [ 6.852751] logrotate[576]: switching euid from 0 to 60 and egid from 0 to 60 (pid 576)837gitlab # [ 6.854047] logrotate[576]: considering log /var/log/nginx/*.log838gitlab # [ 6.855098] logrotate[576]: log /var/log/nginx/*.log does not exist -- skipping839gitlab # [ 6.856711] logrotate[576]: Creating new state840gitlab # [ 6.857366] logrotate[576]: not running postrotate script, since no logs were rotated841gitlab # [ 6.858433] logrotate[576]: switching euid from 60 to 0 and egid from 60 to 0 (pid 576)842gitlab # [ 6.859671] systemd[1]: Starting Virtual Console Setup...843gitlab # [ 6.861614] nsncd[577]: Nov 02 13:10:15.812 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"844gitlab # [ 6.863958] systemd[1]: Started Name Service Cache Daemon (nsncd).845gitlab # [ 6.865190] systemd[1]: Finished Logrotate configuration check.846gitlab # [ 6.866135] systemd[1]: Finished Post-Boot Actions.847gitlab # [ 6.866963] systemd[1]: Finished Import lastlog data into lastlog2 database.848gitlab # [ 6.868007] systemd[1]: Reached target Host and Network Name Lookups.849gitlab # [ 6.868913] systemd[1]: Reached target User and Group Name Lookups.850gitlab # [ 6.869861] systemd[1]: Starting User Login Management...851gitlab # [ 6.870628] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.852gitlab # [ 6.871646] systemd[1]: Stopped Virtual Console Setup.853gitlab # [ 6.881519] systemd[1]: Starting Virtual Console Setup...854gitlab # [ 6.904760] systemd[1]: Stopped target Host and Network Name Lookups.855gitlab # [ 6.905833] systemd[1]: Stopping Host and Network Name Lookups...856gitlab # [ 6.906799] systemd[1]: Stopped target User and Group Name Lookups.857gitlab # [ 6.907679] systemd[1]: Stopping User and Group Name Lookups...858gitlab # [ 6.909614] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...859gitlab # [ 6.912554] systemd[1]: nscd.service: Deactivated successfully.860gitlab # [ 6.915328] systemd[1]: Stopped Name Service Cache Daemon (nsncd).861gitlab # [ 6.918823] systemd[1]: nscd.service: Consumed 27ms CPU time over 122ms wall clock time, 7.7M memory peak, 5.8M read from disk.862gitlab # [ 6.923018] systemd-logind[617]: New seat seat0.863gitlab # [ 6.923742] systemd-logind[617]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)864gitlab # [ 6.925484] systemd-logind[617]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)865gitlab # [ 6.926795] systemd-logind[617]: Watching system buttons on /dev/input/event0 (gpio-keys)866gitlab # [ 6.927958] systemd[1]: Started User Login Management.867gitlab # [ 6.929193] systemd-vconsole-setup[644]: Configuration of first virtual console was skipped, ignoring remaining ones.868gitlab # [ 6.930870] systemd[1]: Starting linger-users.service...869gitlab # [ 6.941043] systemd[1]: Starting Name Service Cache Daemon (nsncd)...870gitlab # [ 6.943968] systemd[1]: Finished Virtual Console Setup.871gitlab # [ 6.949780] systemd[1]: Finished resolvconf update.872gitlab # [ 6.951034] systemd[1]: Reached target Preparation for Network.873gitlab # [ 6.955667] systemd[1]: Starting DHCP Client...874gitlab # [ 6.961895] systemd[1]: Starting Address configuration of eth1...875gitlab # [ 6.964643] nsncd[679]: Nov 02 13:10:15.945 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"876gitlab # [ 6.969044] systemd[1]: Starting Extra networking commands....877gitlab # [ 6.969988] systemd[1]: Started Name Service Cache Daemon (nsncd).878gitlab # [ 6.970862] systemd[1]: linger-users.service: Deactivated successfully.879gitlab # [ 6.971806] systemd[1]: Finished linger-users.service.880gitlab # [ 6.972861] systemd[1]: Reached target Host and Network Name Lookups.881gitlab # [ 6.973858] systemd[1]: Reached target User and Group Name Lookups.882gitlab # [ 6.991955] network-addresses-eth1-start[689]: adding address 192.168.1.1/24... done883gitlab # [ 6.998072] network-addresses-eth1-start[689]: adding address 2001:db8:1::1/64... done884gitlab # [ 7.008959] systemd[1]: Finished Address configuration of eth1.885gitlab # [ 7.033828] dhcpcd[709]: dhcpcd-10.3.2 starting886gitlab # [ 7.035710] dhcpcd[754]: dev: loaded udev887gitlab # [ 7.048202] 8021q: 802.1Q VLAN Support v1.8888gitlab # [ 7.048624] 8021q: adding VLAN 0 to HW filter on device eth1889gitlab # [ 7.057736] systemd[1]: Finished Extra networking commands..890gitlab # [ 7.059338] systemd[1]: Reached target Network.891gitlab # [ 7.060113] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.892gitlab # [ 7.064160] systemd[1]: Starting gitlab-workhorse.service...893gitlab # [ 7.065705] systemd[1]: Starting Nginx Web Server...894gitlab # [ 7.068892] systemd[1]: Starting PostgreSQL Server...895gitlab # [ 7.080112] systemd[1]: Starting Redis Server - redis-gitlab...896gitlab # [ 7.084098] systemd[1]: Starting Permit User Sessions...897gitlab # [ 7.087050] cfg80211: Loading compiled-in X.509 certificates for regulatory database898gitlab # [ 7.093590] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'899gitlab # [ 7.094193] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'900gitlab # [ 7.095913] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2901gitlab # [ 7.096239] cfg80211: failed to load regulatory.db902gitlab # [ 7.110020] systemd[1]: Finished Permit User Sessions.903gitlab # [ 7.118184] systemd[1]: Started Getty on tty1.904gitlab # [ 7.118919] systemd[1]: Reached target Login Prompts.905gitlab # [ 7.131525] 8021q: adding VLAN 0 to HW filter on device eth0906gitlab # [ 7.131033] dhcpcd[754]: eth0: waiting for carrier907gitlab # [ 7.134198] dhcpcd[754]: eth0: carrier acquired908gitlab # [ 7.138964] dhcpcd[754]: DUID 00:01:00:01:32:7b:48:38:52:54:00:12:34:56909gitlab # [ 7.140293] dhcpcd[754]: eth0: IAID 00:12:34:56910gitlab # [ 7.141033] dhcpcd[754]: eth0: adding address fe80::5054:ff:fe12:3456911gitlab # [ 7.241249] nginx-pre-start[802]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok912gitlab # [ 7.248324] nginx-pre-start[802]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful913gitlab # [ 7.250045] systemd[1]: Started Nginx Web Server.914gitlab # [ 7.256979] redis-server[800]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.915gitlab # [ 7.259233] redis-server[800]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo916gitlab # [ 7.264099] redis-server[800]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=800, just started917gitlab # [ 7.267388] systemd[1]: Started Redis Server - redis-gitlab.918gitlab # [ 7.268956] redis-server[800]: Configuration loaded919gitlab # [ 7.271733] redis[800]: monotonic clock: ARM CNTVCT @ 1000 ticks/us920gitlab # [ 7.272964] redis[800]: Running mode=standalone, port=0.921gitlab # [ 7.273952] redis[800]: Server initialized922gitlab # [ 7.274717] redis[800]: Ready to accept connections unix923gitlab # [ 7.275629] redis[800]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.924gitlab # [ 7.283534] postgresql-pre-start[803]: The files belonging to this database system will be owned by user "postgres".925gitlab # [ 7.285467] postgresql-pre-start[803]: This user must also own the server process.926gitlab # [ 7.286584] postgresql-pre-start[803]: The database cluster will be initialized with locale "en_US.UTF-8".927gitlab # [ 7.287869] postgresql-pre-start[803]: The default database encoding has accordingly been set to "UTF8".928gitlab # [ 7.289244] postgresql-pre-start[803]: The default text search configuration will be set to "english".929gitlab # [ 7.290493] postgresql-pre-start[803]: Data page checksums are enabled.930gitlab # [ 7.291384] postgresql-pre-start[803]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok931gitlab # [ 7.294950] postgresql-pre-start[803]: creating subdirectories ... ok932gitlab # [ 7.295879] postgresql-pre-start[803]: selecting dynamic shared memory implementation ... posix933gitlab # [ 7.329408] postgresql-pre-start[803]: selecting default "max_connections" ... 100934gitlab # [ 7.380154] postgresql-pre-start[803]: selecting default "shared_buffers" ... 128MB935gitlab # [ 7.643833] systemd[1]: Started gitlab-workhorse.service.936gitlab # [ 7.865953] postgresql-pre-start[803]: selecting default time zone ... UTC937gitlab # [ 7.867973] postgresql-pre-start[803]: creating configuration files ... ok938gitlab # [ 7.927984] gitlab-workhorse[826]: time="2026-11-02T13:10:16Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4939gitlab # [ 7.930394] gitlab-workhorse[826]: time="2026-11-02T13:10:16Z" level=info msg="Using redis/go-redis"940gitlab # [ 7.931769] gitlab-workhorse[826]: time="2026-11-02T13:10:16Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix941gitlab # [ 8.115344] postgresql-pre-start[803]: running bootstrap script ... ok942gitlab # [ 8.774945] postgresql-pre-start[803]: performing post-bootstrap initialization ... ok943gitlab # [ 8.887879] dhcpcd[754]: eth0: soliciting a DHCP lease944gitlab # [ 8.900647] dhcpcd[754]: eth0: offered 10.0.2.15 from 10.0.2.2945gitlab # [ 8.916324] dhcpcd[754]: eth0: probing address 10.0.2.15/24946gitlab # [ 8.956660] postgresql-pre-start[803]: syncing data to disk ... ok947gitlab # [ 8.958096] postgresql-pre-start[803]: initdb: warning: enabling "trust" authentication for local connections948gitlab # [ 8.960173] postgresql-pre-start[803]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.949gitlab # [ 8.963419] postgresql-pre-start[803]: Success. You can now start the database server using:950gitlab # [ 8.965372] postgresql-pre-start[803]: pg_ctl -D /var/lib/postgresql/18 -l logfile start951gitlab # [ 9.083342] postgres[855]: [855] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit952gitlab # [ 9.086697] postgres[855]: [855] LOG: listening on IPv6 address "::1", port 5432953gitlab # [ 9.088551] postgres[855]: [855] LOG: listening on IPv4 address "127.0.0.1", port 5432954gitlab # [ 9.090373] postgres[855]: [855] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"955gitlab # [ 9.094024] postgres[870]: [870] LOG: database system was shut down at 2026-11-02 13:10:17 GMT956gitlab # [ 9.100077] postgres[855]: [855] LOG: database system is ready to accept connections957gitlab # [ 9.102873] systemd[1]: Started PostgreSQL Server.958gitlab # [ 9.106300] systemd[1]: Starting PostgreSQL Setup Scripts...959gitlab # [ 9.343654] postgresql-setup-start[881]: CREATE DATABASE960gitlab # [ 9.376768] postgresql-setup-start[891]: CREATE ROLE961gitlab # [ 9.390107] postgresql-setup-start[893]: ALTER DATABASE962gitlab # [ 9.419767] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[886]: mkdir -p /var/gitlab/state/home/.ssh: OK963gitlab # [ 9.422160] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[886]: chmod 700 /var/gitlab/state/home/.ssh: OK964gitlab # [ 9.424300] postgresql-setup-start[898]: CREATE ROLE965gitlab # [ 9.425437] systemd[1]: Finished PostgreSQL Setup Scripts.966gitlab # [ 9.426640] systemd[1]: Reached target PostgreSQL.967gitlab # [ 9.429621] systemd[1]: Starting gitlab-postgresql.service...968gitlab # [ 9.484888] systemd[1]: Finished gitlab-config.service.969gitlab # [ 9.487947] systemd[1]: Started gitaly.service.970gitlab # [ 9.491765] systemd[1]: Starting gitlab-db-config.service...971gitlab # [ 9.512244] dhcpcd[754]: eth0: soliciting an IPv6 router972gitlab # [ 9.513544] dhcpcd[754]: eth0: Router Advertisement from fe80::2973gitlab # [ 9.518928] dhcpcd[754]: eth0: adding address fec0::5054:ff:fe12:3456/64974gitlab # [ 9.520351] dhcpcd[754]: eth0: adding route to fec0::/64975gitlab # [ 9.521517] dhcpcd[754]: eth0: adding default route via fe80::2976gitlab # [ 9.564002] gitlab-postgresql-start[917]: CREATE DATABASE977gitlab # [ 9.620936] gitlab-postgresql-start[927]: CREATE EXTENSION978gitlab # [ 9.708435] gitlab-postgresql-start[929]: CREATE EXTENSION979gitlab # [ 9.713187] systemd[1]: Finished gitlab-postgresql.service.980gitlab # [ 10.048485] gitaly[920]: time="2026-11-02T13:10:19.024Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=920981gitlab # [ 10.053351] gitaly[920]: time="2026-11-02T13:10:19.024Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=920982gitlab # [ 10.055406] gitaly[920]: time="2026-11-02T13:10:19.024Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=920983gitlab # [ 10.058148] gitaly[920]: time="2026-11-02T13:10:19.024Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=920984gitlab # [ 10.060395] gitaly[920]: time="2026-11-02T13:10:19.024Z" level=info msg="Starting Gitaly" pid=920 version=985gitlab # [ 10.248618] gitaly[920]: time="2026-11-02T13:10:19.230Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=205 pid=920986gitlab # [ 10.251630] gitaly[920]: time="2026-11-02T13:10:19.233Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=920987gitlab # [ 10.253689] gitaly[920]: time="2026-11-02T13:10:19.233Z" level=info msg="finished initializing command factory" duration_ms=0 pid=920988gitlab # [ 10.255683] gitaly[920]: time="2026-11-02T13:10:19.233Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-920/git-exec-2109028716.d/git pid=920989gitlab # [ 10.262349] gitaly[920]: time="2026-11-02T13:10:19.244Z" level=info msg="finished detecting git version" duration_ms=10 pid=920990gitlab # [ 10.263939] gitaly[920]: time="2026-11-02T13:10:19.244Z" level=info msg="using Git version" pid=920 version=2.53.g991gitlab # [ 10.265795] gitaly[920]: time="2026-11-02T13:10:19.244Z" level=info msg="clearing disk cache object folder" pid=920 storage=default992gitlab # [ 10.267832] gitaly[920]: time="2026-11-02T13:10:19.244Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1696258310 pid=920 storage=default993gitlab # [ 10.271452] gitaly[920]: time="2026-11-02T13:10:19.244Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1696258310 pid=920 storage=default994gitlab # [ 10.275387] gitaly[920]: time="2026-11-02T13:10:19.244Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=920995gitlab # [ 10.278901] gitaly[920]: time="2026-11-02T13:10:19.244Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=920996gitlab # [ 10.282094] gitaly[920]: time="2026-11-02T13:10:19.245Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1696258310 clear_duration_ms=0 pid=920 storage=default997gitlab # [ 10.285971] gitaly[920]: time="2026-11-02T13:10:19.258Z" level=info msg="finished tempdir cleaner walk" pid=920 storage=default time_ms=0998gitlab # [ 10.288103] gitaly[920]: time="2026-11-02T13:10:19.258Z" level=info msg="counting repositories" pid=920999gitlab # [ 10.289787] gitaly[920]: time="2026-11-02T13:10:19.258Z" level=info msg="starting to count repositories" pid=920 storage_path=/var/gitlab/state/repositories1000gitlab # [ 10.292226] gitaly[920]: time="2026-11-02T13:10:19.258Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=920 storage_path=/var/gitlab/state/repositories1001gitlab # [ 10.294834] gitaly[920]: time="2026-11-02T13:10:19.258Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9201002gitlab # [ 10.296941] gitaly[920]: time="2026-11-02T13:10:19.258Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9201003gitlab # [ 10.298956] gitaly[920]: time="2026-11-02T13:10:19.259Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=920 schema=unix1004gitlab # [ 10.300867] gitaly[920]: time="2026-11-02T13:10:19.259Z" level=info msg="listening at address" address=/run/gitaly/gitaly-920/sock.d/intern pid=920 schema=unix1005gitlab # [ 10.303374] gitaly[920]: time="2026-11-02T13:10:19.262Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9201006gitlab # [ 10.306456] gitaly[920]: time="2026-11-02T13:10:19.262Z" level=info msg="Started Gitaly" duration_ms=237 pid=9201007gitlab # [ 10.308290] gitaly[920]: time="2026-11-02T13:10:19.262Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9201008gitlab # [ 10.310448] gitaly[920]: time="2026-11-02T13:10:19.262Z" level=info msg="maintenance: daily scheduled" pid=920 scheduled="2026-11-03 12:00:00 +0000 UTC"1009gitlab # [ 10.595521] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1010gitlab # [ 10.600461] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1011gitlab # [ 10.608330] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1012gitlab # [ 13.747808] gitaly[920]: time="2026-11-02T13:10:22.729Z" level=info msg="License database preloaded" duration_ms=3681 pid=9201013gitlab # [ 14.170895] dhcpcd[754]: eth0: leased 10.0.2.15 for 86400 seconds1014gitlab # [ 14.172414] dhcpcd[754]: eth0: adding route to 10.0.2.0/241015gitlab # [ 14.173655] dhcpcd[754]: eth0: adding default route via 10.0.2.21016gitlab # [ 14.284573] systemd[1]: Started DHCP Client.1017gitlab # [ 14.286810] systemd[1]: Reached target Network is Online.1018gitlab # [ 54.388196] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[923]: W, [2026-11-02T13:11:03.369348 #923] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1019gitlab # [ 57.386817] postgres[1445]: [1445] ERROR: relation "feature_gates" does not exist at character 5231020gitlab # [ 57.388219] postgres[1445]: [1445] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1021gitlab # [ 57.389488] postgres[1445]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1022gitlab # [ 57.390751] postgres[1445]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1023gitlab # [ 57.391920] postgres[1445]: attidentity AS identity,1024gitlab # [ 57.392737] postgres[1445]: attgenerated as attgenerated1025gitlab # [ 57.393570] postgres[1445]: FROM pg_attribute a1026gitlab # [ 57.394249] postgres[1445]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1027gitlab # [ 57.395465] postgres[1445]: LEFT JOIN pg_type t ON a.atttypid = t.oid1028gitlab # [ 57.396409] postgres[1445]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1029gitlab # [ 57.397813] postgres[1445]: WHERE a.attrelid = '"feature_gates"'::regclass1030gitlab # [ 57.398749] postgres[1445]: AND a.attnum > 0 AND NOT a.attisdropped1031gitlab # [ 57.399682] postgres[1445]: ORDER BY a.attnum1032gitlab # [ 57.400340] postgres[1445]:1033gitlab # [ 57.999174] gitlab-workhorse[826]: time="2026-11-02T13:11:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1034gitlab # [ 62.511808] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1496]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1035gitlab # [ 62.514873] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1496]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1036gitlab # [ 68.000225] gitlab-workhorse[826]: time="2026-11-02T13:11:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1037gitlab # [ 78.009661] gitlab-workhorse[826]: time="2026-11-02T13:11:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 82.801535] postgres[1590]: [1590] ERROR: relation "table_without_models" does not exist1039gitlab # [ 82.803402] postgres[1590]: [1590] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1040gitlab # [ 88.017511] gitlab-workhorse[826]: time="2026-11-02T13:11:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1041gitlab # [ 98.018342] gitlab-workhorse[826]: time="2026-11-02T13:11:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1042gitlab # [ 99.958337] systemd[1]: Finished gitlab-db-config.service.1043gitlab # [ 99.961678] systemd[1]: Started gitlab-sidekiq.service.1044gitlab # [ 99.964585] systemd[1]: Starting gitlab.service...1045gitlab # [ 100.518667] bundle[1820]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1046gitlab # [ 100.522070] bundle[1820]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1047gitlab # [ 100.525236] bundle[1820]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1048gitlab # [ 105.255020] sidekiq-cluster[1819]: {"severity":"INFO","time":"2026-11-02T13:11:54.196Z","message":"Starting cluster with 1 processes"}1049gitlab # [ 105.323257] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.275Z","pid":1820,"message":"* Enabling systemd notification integration"}1050gitlab # [ 105.326119] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"Puma starting in cluster mode..."}1051gitlab # [ 105.328531] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1052gitlab # [ 105.330932] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1053gitlab # [ 105.334083] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Min threads: 0"}1054gitlab # [ 105.336090] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Max threads: 4"}1055gitlab # [ 105.338122] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Environment: production"}1056gitlab # [ 105.340353] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Master PID: 1820"}1057gitlab # [ 105.342556] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Workers: 2"}1058gitlab # [ 105.344874] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1059gitlab # [ 105.347675] bundle[1820]: {"timestamp":"2026-11-02T13:11:54.304Z","pid":1820,"message":"* Preloading application"}1060gitlab # [ 108.025465] gitlab-workhorse[826]: time="2026-11-02T13:11:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1061gitlab # [ 118.036119] gitlab-workhorse[826]: time="2026-11-02T13:12:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1062gitlab # [ 128.041041] gitlab-workhorse[826]: time="2026-11-02T13:12:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1063gitlab # [ 138.042945] gitlab-workhorse[826]: time="2026-11-02T13:12:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1064gitlab # [ 144.232721] bundle[1820]: W, [2026-11-02T13:12:33.213947 #1820] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1065gitlab # [ 147.919207] sidekiq-cluster[1873]: 2026-11-02T13:12:36.900Z pid=1873 tid=ws9 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1873, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>423735296}1066gitlab # [ 148.049319] gitlab-workhorse[826]: time="2026-11-02T13:12:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1067gitlab # [ 148.195802] sidekiq-cluster[1873]: W, [2026-11-02T13:12:37.177544 #1873] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1068gitlab # [ 148.340104] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:37.321Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1069gitlab # [ 151.609583] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.590Z","message":"GitLab reliable fetch activated!"}1070gitlab # [ 151.609847] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.591Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1071gitlab # [ 151.642355] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.624Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1072gitlab # [ 151.646111] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.626Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1073gitlab # [ 151.649846] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.629Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1074gitlab # [ 151.653357] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.631Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1075gitlab # [ 151.657232] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.633Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1076gitlab # [ 151.660716] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.636Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1077gitlab # [ 151.664308] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.638Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1078gitlab # [ 151.667521] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.639Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1079gitlab # [ 151.670946] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.641Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1080gitlab # [ 151.674359] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.643Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1081gitlab # [ 151.678177] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.644Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1082gitlab # [ 151.681767] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.647Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1083gitlab # [ 151.685057] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.648Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1084gitlab # [ 151.688536] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.650Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1085gitlab # [ 151.691970] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.652Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1086gitlab # [ 151.695457] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.655Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1087gitlab # [ 151.698591] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.657Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1088gitlab # [ 151.701537] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.660Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1089gitlab # [ 151.704630] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.663Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1090gitlab # [ 151.707420] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.665Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1091gitlab # [ 151.710641] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.668Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1092gitlab # [ 151.713830] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.670Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1093gitlab # [ 151.716532] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.671Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1094gitlab # [ 151.719120] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.674Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1095gitlab # [ 151.721816] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.676Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1096gitlab # [ 151.724316] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.679Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1097gitlab # [ 151.726668] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.682Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1098gitlab # [ 151.729114] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.684Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1099gitlab # [ 151.731496] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.687Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1100gitlab # [ 151.734096] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.690Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1101gitlab # [ 151.736333] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.692Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1102gitlab # [ 151.738709] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.695Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1103gitlab # [ 151.740894] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.697Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1104gitlab # [ 151.743128] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.700Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1105gitlab # [ 151.745399] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.703Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1106gitlab # [ 151.747547] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.705Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1107gitlab # [ 151.749862] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.707Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1108gitlab # [ 151.751936] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.709Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1109gitlab # [ 151.754189] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.712Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1110gitlab # [ 151.756281] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.714Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1111gitlab # [ 151.758504] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.716Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1112gitlab # [ 151.760742] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.718Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1113gitlab # [ 151.762949] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.720Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1114gitlab # [ 151.765219] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.722Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1115gitlab # [ 151.767449] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.725Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1116gitlab # [ 151.769884] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.727Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1117gitlab # [ 151.772135] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.729Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1118gitlab # [ 151.774296] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.731Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1119gitlab # [ 151.776440] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.734Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1120gitlab # [ 151.778564] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.736Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1121gitlab # [ 151.780773] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.739Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1122gitlab # [ 151.782831] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.742Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1123gitlab # [ 151.785003] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.754Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1124gitlab # [ 151.787335] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.757Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1125gitlab # [ 151.789646] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.759Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1126gitlab # [ 151.791833] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.762Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1127gitlab # [ 151.794180] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.764Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1128gitlab # [ 151.861500] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.841Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1129gitlab # [ 151.865140] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.845Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1130gitlab # [ 151.868846] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.848Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1131gitlab # [ 151.872105] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.850Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1132gitlab # [ 151.875340] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.853Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1133gitlab # [ 151.878928] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.856Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1134gitlab # [ 151.882515] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.859Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1135gitlab # [ 151.886177] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.862Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1136gitlab # [ 151.889753] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.865Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1137gitlab # [ 151.893182] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.868Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1138gitlab # [ 151.896635] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.870Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1139gitlab # [ 151.900288] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.872Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1140gitlab # [ 151.903951] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.874Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1141gitlab # [ 151.907537] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.876Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1142gitlab # [ 151.910996] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.879Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1143gitlab # [ 151.914246] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.881Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1144gitlab # [ 151.917535] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.882Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1145gitlab # [ 151.920626] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.885Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1146gitlab # [ 151.923617] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.887Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1147gitlab # [ 151.926519] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.890Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1148gitlab # [ 151.929604] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.891Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1149gitlab # [ 151.932272] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.894Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1150gitlab # [ 151.934705] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.895Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1151gitlab # [ 151.937495] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.898Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1152gitlab # [ 151.940325] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.899Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1153gitlab # [ 151.942859] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.902Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1154gitlab # [ 151.945356] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.904Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1155gitlab # [ 151.947884] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.907Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1156gitlab # [ 151.950349] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.909Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1157gitlab # [ 151.952826] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.911Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1158gitlab # [ 151.955087] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.913Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1159gitlab # [ 151.957581] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.915Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1160gitlab # [ 151.959867] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.917Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1161gitlab # [ 151.962348] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.919Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1162gitlab # [ 151.964734] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.920Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1163gitlab # [ 151.967061] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.922Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1164gitlab # [ 151.969621] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.924Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1165gitlab # [ 151.971673] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.927Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1166gitlab # [ 151.974210] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.930Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1167gitlab # [ 151.976531] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.932Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1168gitlab # [ 151.978780] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.934Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1169gitlab # [ 151.981138] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.948Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1170gitlab # [ 151.983408] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.950Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1171gitlab # [ 151.985758] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.951Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1172gitlab # [ 151.988116] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.955Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1173gitlab # [ 151.990222] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.958Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1174gitlab # [ 151.992501] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.961Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1175gitlab # [ 151.994897] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.964Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1176gitlab # [ 151.997431] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.966Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1177gitlab # [ 151.999573] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.967Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1178gitlab # [ 152.002004] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.971Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1179gitlab # [ 152.004332] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:12:40.973Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1180gitlab # [ 158.059208] gitlab-workhorse[826]: time="2026-11-02T13:12:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1181gitlab # [ 168.070075] gitlab-workhorse[826]: time="2026-11-02T13:12:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1182gitlab # [ 178.077572] gitlab-workhorse[826]: time="2026-11-02T13:13:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1183gitlab # [ 188.085536] gitlab-workhorse[826]: time="2026-11-02T13:13:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1184gitlab # [ 188.268333] postgres[2615]: [2615] ERROR: relation "table_without_models" does not exist1185gitlab # [ 188.270232] postgres[2615]: [2615] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1186gitlab # [ 193.090052] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:22.071Z","message":"Booted Rails 7.2.3.1 application in production environment"}1187gitlab # [ 193.092805] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:22.074Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1188gitlab # [ 193.095380] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:22.076Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1189gitlab # [ 193.097322] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:22.077Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1190gitlab # [ 193.099355] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:22.080Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1191gitlab # [ 193.384354] postgres[2663]: [2663] ERROR: relation "table_without_models" does not exist1192gitlab # [ 193.386198] postgres[2663]: [2663] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1193gitlab # [ 195.802138] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:24.783Z","message":"Cleaning working queues"}1194gitlab # [ 195.805760] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:24.784Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"80f08457635e4e62d4406e5b","created_at":"2026-11-02T13:11:48.335Z","trace_propagation_headers":{"sentry-trace":"4f75ba871c9943ea85277c7d7a3916f8-52895cb0a1c84730","baggage":"sentry-trace_id=4f75ba871c9943ea85277c7d7a3916f8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"8eb476e087831dc1149fba187a801ab7","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-02T13:11:48.347Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-80f08457635e4e62d4406e5b: start","job_status":"start","queue_duration_s":96.437264,"scheduling_latency_s":96.437315}1195gitlab # [ 196.033924] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:25.015Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"80f08457635e4e62d4406e5b","created_at":"2026-11-02T13:11:48.335Z","trace_propagation_headers":{"sentry-trace":"4f75ba871c9943ea85277c7d7a3916f8-52895cb0a1c84730","baggage":"sentry-trace_id=4f75ba871c9943ea85277c7d7a3916f8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"8eb476e087831dc1149fba187a801ab7","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-02T13:11:48.347Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-80f08457635e4e62d4406e5b: done: 0.230717 sec","job_status":"done","queue_duration_s":96.437264,"scheduling_latency_s":96.437315,"redis_calls":29,"redis_duration_s":0.006428,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.003698,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001672,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12312569999999999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.230717,"completed_at":"2026-11-02T13:13:25.015Z","load_balancing_strategy":"primary","db_duration_s":0.059393,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1196gitlab # [ 198.093424] gitlab-workhorse[826]: time="2026-11-02T13:13:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1197gitlab # [ 199.874709] bundle[1820]: {"timestamp":"2026-11-02T13:13:28.855Z","pid":1820,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1198gitlab # [ 199.875016] bundle[1820]: {"timestamp":"2026-11-02T13:13:28.856Z","pid":1820,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1199gitlab # [ 199.875136] bundle[1820]: {"timestamp":"2026-11-02T13:13:28.856Z","pid":1820,"message":"! #\u003cThread:0x0000fb58824f5d40 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1200gitlab # [ 199.875204] bundle[1820]: {"timestamp":"2026-11-02T13:13:28.856Z","pid":1820,"message":"! #\u003cThread:0x0000fb58826f1658 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1201gitlab # [ 199.875264] bundle[1820]: {"timestamp":"2026-11-02T13:13:28.856Z","pid":1820,"message":"Use Ctrl-C to stop"}1202gitlab # [ 199.875321] bundle[1820]: {"timestamp":"2026-11-02T13:13:28.856Z","pid":1820,"message":"Sending status to systemd every 1.0 sec"}1203gitlab # [ 200.427918] bundle[1820]: {"timestamp":"2026-11-02T13:13:29.409Z","pid":1820,"message":"- Worker 0 (PID: 2763) booted in 0.52s, phase: 0"}1204gitlab # [ 200.444156] bundle[1820]: {"timestamp":"2026-11-02T13:13:29.425Z","pid":1820,"message":"- Worker 1 (PID: 2767) booted in 0.51s, phase: 0"}1205gitlab # [ 200.449519] systemd[1]: Started gitlab.service.1206gitlab # [ 200.450535] systemd[1]: Reached target Common target for all GitLab services..1207gitlab # [ 200.453491] systemd[1]: Starting setup-gitlab.service...1208gitlab # [ 200.479287] setup-gitlab-start[2791]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1209gitlab # [ 204.337198] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:13:33 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 38311210gitlab # [ 204.340631] setup-gitlab-start[2791]: + TOKEN=nixbot-test-token-of-20-chars1211gitlab # [ 204.342222] setup-gitlab-start[2791]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1212gitlab # [ 204.351744] runuser[2817]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1213gitlab # [ 204.881418] setup-gitlab-start[2818]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1214gitlab # [ 204.885070] setup-gitlab-start[2818]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1215gitlab # [ 204.888334] setup-gitlab-start[2818]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1216gitlab # [ 205.710299] setup-gitlab-start[2818]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1217gitlab # [ 205.710709] setup-gitlab-start[2818]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1218gitlab # [ 205.717369] setup-gitlab-start[2818]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1219gitlab # [ 209.516926] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:38.497Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"914a5863ebd4a8ed04d27af9","created_at":"2026-11-02T13:13:38.433Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d7dcdc74f685c1d349fe9ae6f6a3c95","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.495Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-914a5863ebd4a8ed04d27af9: start","job_status":"start","queue_duration_s":0.002161,"scheduling_latency_s":0.002203}1220gitlab # [ 209.634206] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:38.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"914a5863ebd4a8ed04d27af9","created_at":"2026-11-02T13:13:38.433Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d7dcdc74f685c1d349fe9ae6f6a3c95","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.495Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-914a5863ebd4a8ed04d27af9: done: 0.117577 sec","job_status":"done","queue_duration_s":0.002161,"scheduling_latency_s":0.002203,"redis_calls":12,"redis_duration_s":0.02234,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.017272,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002233,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042618,"duration_s":0.117577,"completed_at":"2026-11-02T13:13:38.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1221gitlab # [ 209.700724] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:38.618Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0263d5c6f7608b5afe79e1d5","created_at":"2026-11-02T13:13:38.513Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40d3203ca7cd6e5e4939726b76b2b55e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.523Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0263d5c6f7608b5afe79e1d5: start","job_status":"start","queue_duration_s":0.094579,"scheduling_latency_s":0.094607}1222gitlab # [ 210.347503] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.328Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0263d5c6f7608b5afe79e1d5","created_at":"2026-11-02T13:13:38.513Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40d3203ca7cd6e5e4939726b76b2b55e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.523Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0263d5c6f7608b5afe79e1d5: done: 0.710489 sec","job_status":"done","queue_duration_s":0.094579,"scheduling_latency_s":0.094607,"redis_calls":14,"redis_duration_s":0.003824,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001579,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.096,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.094,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.345369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00086468,"exclusive_lock_hold_duration_s":0.64272164,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.710489,"completed_at":"2026-11-02T13:13:39.328Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.263661,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1223gitlab # [ 210.421893] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7e0b162716d3c5d176e730ef","created_at":"2026-11-02T13:13:38.552Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea374124fc11172799c0abbcd135b8d9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.559Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7e0b162716d3c5d176e730ef: start","job_status":"start","queue_duration_s":0.843749,"scheduling_latency_s":0.843784}1224gitlab # [ 210.468153] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7e0b162716d3c5d176e730ef","created_at":"2026-11-02T13:13:38.552Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea374124fc11172799c0abbcd135b8d9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.559Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7e0b162716d3c5d176e730ef: done: 0.045862 sec","job_status":"done","queue_duration_s":0.843749,"scheduling_latency_s":0.843784,"redis_calls":10,"redis_duration_s":0.003136,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000809,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000682,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006809,"duration_s":0.045862,"completed_at":"2026-11-02T13:13:39.449Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006838,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1225gitlab # [ 210.530661] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f5d99e3ee5b8a3aa6d6c6539","created_at":"2026-11-02T13:13:38.572Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe005cc9ce6152185d15b5c18d6b0793","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-02T13:13:38.580Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f5d99e3ee5b8a3aa6d6c6539: start","job_status":"start","queue_duration_s":0.871096,"scheduling_latency_s":0.871125}1226gitlab # [ 210.545966] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.468Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1227gitlab # [ 210.550049] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f5d99e3ee5b8a3aa6d6c6539","created_at":"2026-11-02T13:13:38.572Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe005cc9ce6152185d15b5c18d6b0793","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-02T13:13:38.580Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f5d99e3ee5b8a3aa6d6c6539: done: 0.02623 sec","job_status":"done","queue_duration_s":0.871096,"scheduling_latency_s":0.871125,"redis_calls":10,"redis_duration_s":0.002367,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000557,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001408,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033082,"duration_s":0.02623,"completed_at":"2026-11-02T13:13:39.477Z","load_balancing_strategy":"primary","db_duration_s":0.000471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1228gitlab # [ 210.592422] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"99200d39e9ab5c7550c203a5","created_at":"2026-11-02T13:13:38.586Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"340eb0e1ed37732e4bbaa01f387296f9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.612Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-99200d39e9ab5c7550c203a5: start","job_status":"start","queue_duration_s":0.866748,"scheduling_latency_s":0.866768}1229gitlab # [ 210.607495] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"99200d39e9ab5c7550c203a5","created_at":"2026-11-02T13:13:38.586Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"340eb0e1ed37732e4bbaa01f387296f9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.612Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-99200d39e9ab5c7550c203a5: done: 0.034859 sec","job_status":"done","queue_duration_s":0.866748,"scheduling_latency_s":0.866768,"redis_calls":13,"redis_duration_s":0.00227,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000343,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001165,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01179698,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.034859,"completed_at":"2026-11-02T13:13:39.513Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002946,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1230gitlab # [ 210.654314] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.515Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6c5d56d8c3ea5f38629ad235","created_at":"2026-11-02T13:13:38.631Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7774bbc4c823c4ca9d3740507548418","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-02T13:13:38.639Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6c5d56d8c3ea5f38629ad235: start","job_status":"start","queue_duration_s":0.876027,"scheduling_latency_s":0.876046}1231gitlab # [ 210.668278] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.527Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1232gitlab # [ 210.671713] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6c5d56d8c3ea5f38629ad235","created_at":"2026-11-02T13:13:38.631Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7774bbc4c823c4ca9d3740507548418","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-02T13:13:38.639Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6c5d56d8c3ea5f38629ad235: done: 0.019469 sec","job_status":"done","queue_duration_s":0.876027,"scheduling_latency_s":0.876046,"redis_calls":10,"redis_duration_s":0.001766,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000372,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029094,"duration_s":0.019469,"completed_at":"2026-11-02T13:13:39.534Z","load_balancing_strategy":"primary","db_duration_s":0.000301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1233gitlab # [ 210.714411] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"e11c4d7107f6d74cb5a44153","created_at":"2026-11-02T13:13:38.672Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ba6f3318abb5e5b50b4af7802fe0314","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.707Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-e11c4d7107f6d74cb5a44153: start","job_status":"start","queue_duration_s":0.828345,"scheduling_latency_s":0.828361}1234gitlab # [ 210.728066] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.565Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"e11c4d7107f6d74cb5a44153","created_at":"2026-11-02T13:13:38.672Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ba6f3318abb5e5b50b4af7802fe0314","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.707Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-e11c4d7107f6d74cb5a44153: done: 0.02897 sec","job_status":"done","queue_duration_s":0.828345,"scheduling_latency_s":0.828361,"redis_calls":12,"redis_duration_s":0.001958,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00071,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049254,"exclusive_lock_hold_duration_s":0.01055622,"duration_s":0.02897,"completed_at":"2026-11-02T13:13:39.565Z","load_balancing_strategy":"primary","db_duration_s":0.003185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1235gitlab # [ 210.770605] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.566Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d76528f68ed8c3356c4537e9","created_at":"2026-11-02T13:13:38.778Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a242e4c26950a51696b1601495a1414","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.785Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d76528f68ed8c3356c4537e9: start","job_status":"start","queue_duration_s":0.781484,"scheduling_latency_s":0.7815}1236gitlab # [ 210.785579] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d76528f68ed8c3356c4537e9","created_at":"2026-11-02T13:13:38.778Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a242e4c26950a51696b1601495a1414","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.785Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d76528f68ed8c3356c4537e9: done: 0.02264 sec","job_status":"done","queue_duration_s":0.781484,"scheduling_latency_s":0.7815,"redis_calls":12,"redis_duration_s":0.002125,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000411,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050112,"exclusive_lock_hold_duration_s":0.00329008,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.02264,"completed_at":"2026-11-02T13:13:39.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1237gitlab # [ 210.831356] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c8ea22621fce50f029375cd3","created_at":"2026-11-02T13:13:38.788Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e883399e8dca80c8c10d1c5e0462cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.794Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c8ea22621fce50f029375cd3: start","job_status":"start","queue_duration_s":0.796036,"scheduling_latency_s":0.796056}1238gitlab # [ 210.846847] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c8ea22621fce50f029375cd3","created_at":"2026-11-02T13:13:38.788Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81e883399e8dca80c8c10d1c5e0462cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.794Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c8ea22621fce50f029375cd3: done: 0.056547 sec","job_status":"done","queue_duration_s":0.796036,"scheduling_latency_s":0.796056,"redis_calls":12,"redis_duration_s":0.002745,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001202,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070644,"exclusive_lock_hold_duration_s":0.018761820000000002,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.056547,"completed_at":"2026-11-02T13:13:39.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1239gitlab # [ 210.893807] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"503ee698f7efab71ad8896e1","created_at":"2026-11-02T13:13:38.814Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.821Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-503ee698f7efab71ad8896e1: start","job_status":"start","queue_duration_s":0.827601,"scheduling_latency_s":0.827618}1240gitlab # [ 210.907861] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"503ee698f7efab71ad8896e1","created_at":"2026-11-02T13:13:38.814Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.821Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-503ee698f7efab71ad8896e1: done: 0.038165 sec","job_status":"done","queue_duration_s":0.827601,"scheduling_latency_s":0.827618,"redis_calls":19,"redis_duration_s":0.003627,"redis_read_bytes":22,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000931,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000346,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001681,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000669,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025388,"duration_s":0.038165,"completed_at":"2026-11-02T13:13:39.687Z","load_balancing_strategy":"primary","db_duration_s":0.002124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1241gitlab # [ 210.951693] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c5332b4094b090f57d119744","created_at":"2026-11-02T13:13:38.854Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50e92ef542996a8b7d3a8fe49df4f43b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.869Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c5332b4094b090f57d119744: start","job_status":"start","queue_duration_s":0.819176,"scheduling_latency_s":0.819194}1242gitlab # [ 210.966820] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c5332b4094b090f57d119744","created_at":"2026-11-02T13:13:38.854Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50e92ef542996a8b7d3a8fe49df4f43b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.869Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c5332b4094b090f57d119744: done: 0.039971 sec","job_status":"done","queue_duration_s":0.819176,"scheduling_latency_s":0.819194,"redis_calls":12,"redis_duration_s":0.00228,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060186,"exclusive_lock_hold_duration_s":0.01932888,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.039971,"completed_at":"2026-11-02T13:13:39.728Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 211.012864] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.730Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bab4f8ceecb3ee8d735872c1","created_at":"2026-11-02T13:13:38.892Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab8a6469ff1e865a81a75e1482c02e55","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.910Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bab4f8ceecb3ee8d735872c1: start","job_status":"start","queue_duration_s":0.819585,"scheduling_latency_s":0.819604}1244gitlab # [ 211.028165] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bab4f8ceecb3ee8d735872c1","created_at":"2026-11-02T13:13:38.892Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab8a6469ff1e865a81a75e1482c02e55","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.910Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bab4f8ceecb3ee8d735872c1: done: 0.041131 sec","job_status":"done","queue_duration_s":0.819585,"scheduling_latency_s":0.819604,"redis_calls":10,"redis_duration_s":0.002084,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034502,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.041131,"completed_at":"2026-11-02T13:13:39.771Z","load_balancing_strategy":"primary","db_duration_s":0.016136,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1245gitlab # [ 211.071843] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c9373e280f241e5bfa4d4849","created_at":"2026-11-02T13:13:38.923Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce8a479d34a45800399f0e958a986959","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.941Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c9373e280f241e5bfa4d4849: start","job_status":"start","queue_duration_s":0.831568,"scheduling_latency_s":0.831587}1246gitlab # [ 211.087111] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.785Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1247gitlab # [ 211.090710] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c9373e280f241e5bfa4d4849","created_at":"2026-11-02T13:13:38.923Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce8a479d34a45800399f0e958a986959","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.941Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c9373e280f241e5bfa4d4849: done: 0.019568 sec","job_status":"done","queue_duration_s":0.831568,"scheduling_latency_s":0.831587,"redis_calls":10,"redis_duration_s":0.001667,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000363,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025686,"duration_s":0.019568,"completed_at":"2026-11-02T13:13:39.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1248gitlab # [ 211.134416] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d54073cdeaf0c986281a71da","created_at":"2026-11-02T13:13:38.960Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6cd61f631226485a8bb2e91b02667aae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.974Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d54073cdeaf0c986281a71da: start","job_status":"start","queue_duration_s":0.819864,"scheduling_latency_s":0.819879}1249gitlab # [ 211.149516] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d54073cdeaf0c986281a71da","created_at":"2026-11-02T13:13:38.960Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6cd61f631226485a8bb2e91b02667aae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:38.974Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d54073cdeaf0c986281a71da: done: 0.018989 sec","job_status":"done","queue_duration_s":0.819864,"scheduling_latency_s":0.819879,"redis_calls":10,"redis_duration_s":0.001669,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024928,"duration_s":0.018989,"completed_at":"2026-11-02T13:13:39.812Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1250gitlab # [ 211.192929] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:39.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0ac3a1c859140e2b73491ba3","created_at":"2026-11-02T13:13:38.996Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e5ac7939d7129b9b433a262de8ca1df","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.009Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0ac3a1c859140e2b73491ba3: start","job_status":"start","queue_duration_s":0.805057,"scheduling_latency_s":0.805076}1251gitlab # [ 211.207161] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.142Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0ac3a1c859140e2b73491ba3","created_at":"2026-11-02T13:13:38.996Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e5ac7939d7129b9b433a262de8ca1df","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.009Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0ac3a1c859140e2b73491ba3: done: 0.327894 sec","job_status":"done","queue_duration_s":0.805057,"scheduling_latency_s":0.805076,"redis_calls":565,"redis_duration_s":0.07033299999999999,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.069034,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.290135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023726,"duration_s":0.327894,"completed_at":"2026-11-02T13:13:40.142Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1252gitlab # [ 211.251631] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a75019813faa42df49f3136e","created_at":"2026-11-02T13:13:39.028Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22d7eee5dcc89cdbd762cac83c4ed76d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.041Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a75019813faa42df49f3136e: start","job_status":"start","queue_duration_s":1.102037,"scheduling_latency_s":1.102054}1253gitlab # [ 211.266882] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.155Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1254gitlab # [ 211.271055] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a75019813faa42df49f3136e","created_at":"2026-11-02T13:13:39.028Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22d7eee5dcc89cdbd762cac83c4ed76d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.041Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a75019813faa42df49f3136e: done: 0.019298 sec","job_status":"done","queue_duration_s":1.102037,"scheduling_latency_s":1.102054,"redis_calls":10,"redis_duration_s":0.001501,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000305,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000295,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025774,"duration_s":0.019298,"completed_at":"2026-11-02T13:13:40.163Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000521,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1255gitlab # [ 211.314721] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f6cbc8a7b6dae2af298dc910","created_at":"2026-11-02T13:13:39.076Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d8ad8f6178e67da71492c9f1cd93c01","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.094Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f6cbc8a7b6dae2af298dc910: start","job_status":"start","queue_duration_s":1.069604,"scheduling_latency_s":1.069622}1256gitlab # [ 211.329712] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.176Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1257gitlab # [ 211.333075] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f6cbc8a7b6dae2af298dc910","created_at":"2026-11-02T13:13:39.076Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d8ad8f6178e67da71492c9f1cd93c01","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.094Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f6cbc8a7b6dae2af298dc910: done: 0.018664 sec","job_status":"done","queue_duration_s":1.069604,"scheduling_latency_s":1.069622,"redis_calls":10,"redis_duration_s":0.001821,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000332,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000338,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025686,"duration_s":0.018664,"completed_at":"2026-11-02T13:13:40.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 211.376329] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"66f5cccfd04512f89d388dfd","created_at":"2026-11-02T13:13:39.109Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9f775a6495ccb563ef8307ff6837721","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.128Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-66f5cccfd04512f89d388dfd: start","job_status":"start","queue_duration_s":1.056234,"scheduling_latency_s":1.056249}1259gitlab # [ 211.391092] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.195Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1260gitlab # [ 211.394851] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"66f5cccfd04512f89d388dfd","created_at":"2026-11-02T13:13:39.109Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9f775a6495ccb563ef8307ff6837721","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.128Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-66f5cccfd04512f89d388dfd: done: 0.018918 sec","job_status":"done","queue_duration_s":1.056234,"scheduling_latency_s":1.056249,"redis_calls":10,"redis_duration_s":0.00182,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00037,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000396,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025414,"duration_s":0.018918,"completed_at":"2026-11-02T13:13:40.203Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1261gitlab # [ 211.439494] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d534ead0c78ac5f50414a395","created_at":"2026-11-02T13:13:39.149Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8593e9f4b10f5fa81941ac86db131383","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-02T13:13:39.162Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d534ead0c78ac5f50414a395: start","job_status":"start","queue_duration_s":1.042338,"scheduling_latency_s":1.042356}1262gitlab # [ 211.453024] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.237Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d534ead0c78ac5f50414a395","created_at":"2026-11-02T13:13:39.149Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8593e9f4b10f5fa81941ac86db131383","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-02T13:13:39.162Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d534ead0c78ac5f50414a395: done: 0.033241 sec","job_status":"done","queue_duration_s":1.042338,"scheduling_latency_s":1.042356,"redis_calls":10,"redis_duration_s":0.0018599999999999999,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000338,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026264,"duration_s":0.033241,"completed_at":"2026-11-02T13:13:40.237Z","load_balancing_strategy":"primary","db_duration_s":0.004648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1263gitlab # [ 211.495083] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.239Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b2de33fdec9a7b58592bc9c8","created_at":"2026-11-02T13:13:39.190Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad1a3a69be8bfa2ad0a7f9155d0aa360","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.207Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b2de33fdec9a7b58592bc9c8: start","job_status":"start","queue_duration_s":1.031712,"scheduling_latency_s":1.031727}1264gitlab # [ 211.509766] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.272Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b2de33fdec9a7b58592bc9c8","created_at":"2026-11-02T13:13:39.190Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad1a3a69be8bfa2ad0a7f9155d0aa360","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.207Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b2de33fdec9a7b58592bc9c8: done: 0.033629 sec","job_status":"done","queue_duration_s":1.031712,"scheduling_latency_s":1.031727,"redis_calls":16,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001581,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025302,"duration_s":0.033629,"completed_at":"2026-11-02T13:13:40.272Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005465,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1265gitlab # [ 211.553486] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.274Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e6abd08a63350461c60c7259","created_at":"2026-11-02T13:13:39.219Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.237Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e6abd08a63350461c60c7259: start","job_status":"start","queue_duration_s":1.036804,"scheduling_latency_s":1.036819}1266gitlab # [ 211.567664] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"e6abd08a63350461c60c7259","created_at":"2026-11-02T13:13:39.219Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.237Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-e6abd08a63350461c60c7259: done: 0.21643 sec","job_status":"done","queue_duration_s":1.036804,"scheduling_latency_s":1.036819,"redis_calls":71,"redis_duration_s":0.012721,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001761,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00073,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009628,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000602,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.202307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034596,"duration_s":0.21643,"completed_at":"2026-11-02T13:13:40.490Z","load_balancing_strategy":"primary","db_duration_s":0.001853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1267gitlab # [ 211.611946] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b30afec26b6ca19b368a3f8a","created_at":"2026-11-02T13:13:39.251Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ab64758839d361ee1bd88b073dcb012","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.268Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b30afec26b6ca19b368a3f8a: start","job_status":"start","queue_duration_s":1.223182,"scheduling_latency_s":1.223198}1268gitlab # [ 211.626137] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b30afec26b6ca19b368a3f8a","created_at":"2026-11-02T13:13:39.251Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ab64758839d361ee1bd88b073dcb012","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.268Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b30afec26b6ca19b368a3f8a: done: 0.027081 sec","job_status":"done","queue_duration_s":1.223182,"scheduling_latency_s":1.223198,"redis_calls":12,"redis_duration_s":0.002103,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000808,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049304,"exclusive_lock_hold_duration_s":0.008079500000000002,"duration_s":0.027081,"completed_at":"2026-11-02T13:13:40.518Z","load_balancing_strategy":"primary","db_duration_s":0.003617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1269gitlab # [ 211.670027] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.520Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"428966782f97744295fc4011","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-428966782f97744295fc4011: start","job_status":"start","queue_duration_s":0.840075,"scheduling_latency_s":0.84009}1270gitlab # [ 211.682721] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.536Z","message":"No existing merge request to be cleaned up."}1271gitlab # [ 211.684504] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.549Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"428966782f97744295fc4011","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-428966782f97744295fc4011: done: 0.029642 sec","job_status":"done","queue_duration_s":0.840075,"scheduling_latency_s":0.84009,"redis_calls":13,"redis_duration_s":0.0024319999999999997,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000364,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025288,"duration_s":0.029642,"completed_at":"2026-11-02T13:13:40.549Z","load_balancing_strategy":"primary","db_duration_s":0.003982,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1272gitlab # [ 211.725975] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.551Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6eb73ed570f3c4f448fb0fdb","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6eb73ed570f3c4f448fb0fdb: start","job_status":"start","queue_duration_s":0.871002,"scheduling_latency_s":0.871018}1273gitlab # [ 211.738684] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.562Z","message":"No existing merge request to be cleaned up."}1274gitlab # [ 211.740470] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.570Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6eb73ed570f3c4f448fb0fdb","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6eb73ed570f3c4f448fb0fdb: done: 0.019024 sec","job_status":"done","queue_duration_s":0.871002,"scheduling_latency_s":0.871018,"redis_calls":10,"redis_duration_s":0.001739,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019024,"completed_at":"2026-11-02T13:13:40.569Z","load_balancing_strategy":"primary","db_duration_s":0.001813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1275gitlab # [ 211.780066] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.571Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f55a24a0b6a9599719be45f7","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f55a24a0b6a9599719be45f7: start","job_status":"start","queue_duration_s":0.891211,"scheduling_latency_s":0.891226}1276gitlab # [ 211.792646] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.581Z","message":"No existing merge request to be cleaned up."}1277gitlab # [ 211.794376] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.591Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f55a24a0b6a9599719be45f7","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f55a24a0b6a9599719be45f7: done: 0.020036 sec","job_status":"done","queue_duration_s":0.891211,"scheduling_latency_s":0.891226,"redis_calls":10,"redis_duration_s":0.0017699999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020036,"completed_at":"2026-11-02T13:13:40.591Z","load_balancing_strategy":"primary","db_duration_s":0.001797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1278gitlab # [ 211.835091] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.592Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d2345f3e3726225ede939381","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d2345f3e3726225ede939381: start","job_status":"start","queue_duration_s":0.91243,"scheduling_latency_s":0.912446}1279gitlab # [ 211.847906] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.603Z","message":"No existing merge request to be cleaned up."}1280gitlab # [ 211.849734] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.610Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d2345f3e3726225ede939381","created_at":"2026-11-02T13:13:39.666Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-d074f105dbeb44ab","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7908df9db3ce1dbe61eb0c45557f357c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:39.680Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d2345f3e3726225ede939381: done: 0.018279 sec","job_status":"done","queue_duration_s":0.91243,"scheduling_latency_s":0.912446,"redis_calls":10,"redis_duration_s":0.0016979999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018279,"completed_at":"2026-11-02T13:13:40.610Z","load_balancing_strategy":"primary","db_duration_s":0.001681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1281gitlab # [ 211.889370] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2856e5f5da3fc2c58504223c","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2856e5f5da3fc2c58504223c: start","job_status":"start","queue_duration_s":0.129952,"scheduling_latency_s":0.129967}1282gitlab # [ 211.902254] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2856e5f5da3fc2c58504223c","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2856e5f5da3fc2c58504223c: done: 0.022173 sec","job_status":"done","queue_duration_s":0.129952,"scheduling_latency_s":0.129967,"redis_calls":16,"redis_duration_s":0.002552,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001417,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025338,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022173,"completed_at":"2026-11-02T13:13:40.634Z","load_balancing_strategy":"primary","db_duration_s":0.000488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1283gitlab # [ 211.944600] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60be09661f85f4451f94e2e8","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60be09661f85f4451f94e2e8: start","job_status":"start","queue_duration_s":0.153313,"scheduling_latency_s":0.153328}1284gitlab # [ 211.957078] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60be09661f85f4451f94e2e8","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60be09661f85f4451f94e2e8: done: 0.015091 sec","job_status":"done","queue_duration_s":0.153313,"scheduling_latency_s":0.153328,"redis_calls":13,"redis_duration_s":0.002017,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015091,"completed_at":"2026-11-02T13:13:40.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1285gitlab # [ 211.997244] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa1005ace0435bb1dc73cccb","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa1005ace0435bb1dc73cccb: start","job_status":"start","queue_duration_s":0.17015,"scheduling_latency_s":0.170166}1286gitlab # [ 212.009839] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa1005ace0435bb1dc73cccb","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa1005ace0435bb1dc73cccb: done: 0.014855 sec","job_status":"done","queue_duration_s":0.17015,"scheduling_latency_s":0.170166,"redis_calls":13,"redis_duration_s":0.0018349999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014855,"completed_at":"2026-11-02T13:13:40.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1287gitlab # [ 212.050957] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78170a5f5da5758ed33d26d2","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78170a5f5da5758ed33d26d2: start","job_status":"start","queue_duration_s":0.186122,"scheduling_latency_s":0.186137}1288gitlab # [ 212.063735] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78170a5f5da5758ed33d26d2","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78170a5f5da5758ed33d26d2: done: 0.014842 sec","job_status":"done","queue_duration_s":0.186122,"scheduling_latency_s":0.186137,"redis_calls":13,"redis_duration_s":0.001882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014842,"completed_at":"2026-11-02T13:13:40.682Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1289gitlab # [ 212.104280] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a688be6aace50f9dfa33e66f","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a688be6aace50f9dfa33e66f: start","job_status":"start","queue_duration_s":0.202102,"scheduling_latency_s":0.202117}1290gitlab # [ 212.116740] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a688be6aace50f9dfa33e66f","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a688be6aace50f9dfa33e66f: done: 0.014542 sec","job_status":"done","queue_duration_s":0.202102,"scheduling_latency_s":0.202117,"redis_calls":13,"redis_duration_s":0.001835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014542,"completed_at":"2026-11-02T13:13:40.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 212.157025] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ecd057c517356b167ba8031","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ecd057c517356b167ba8031: start","job_status":"start","queue_duration_s":0.217824,"scheduling_latency_s":0.21784}1292gitlab # [ 212.169508] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ecd057c517356b167ba8031","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ecd057c517356b167ba8031: done: 0.014745 sec","job_status":"done","queue_duration_s":0.217824,"scheduling_latency_s":0.21784,"redis_calls":13,"redis_duration_s":0.001832,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014745,"completed_at":"2026-11-02T13:13:40.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1293gitlab # [ 212.209524] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3dd77eb687eabb57bba7f6","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3dd77eb687eabb57bba7f6: start","job_status":"start","queue_duration_s":0.234325,"scheduling_latency_s":0.234341}1294gitlab # [ 212.222337] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3dd77eb687eabb57bba7f6","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3dd77eb687eabb57bba7f6: done: 0.014532 sec","job_status":"done","queue_duration_s":0.234325,"scheduling_latency_s":0.234341,"redis_calls":13,"redis_duration_s":0.001746,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014532,"completed_at":"2026-11-02T13:13:40.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1295gitlab # [ 212.263694] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"216f3b3b2e026ecf7fe6187e","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-216f3b3b2e026ecf7fe6187e: start","job_status":"start","queue_duration_s":0.24997,"scheduling_latency_s":0.249985}1296gitlab # [ 212.276654] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"216f3b3b2e026ecf7fe6187e","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-216f3b3b2e026ecf7fe6187e: done: 0.017913 sec","job_status":"done","queue_duration_s":0.24997,"scheduling_latency_s":0.249985,"redis_calls":13,"redis_duration_s":0.002038,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017913,"completed_at":"2026-11-02T13:13:40.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1297gitlab # [ 212.317040] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f3c4d02706db1456931d35a","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f3c4d02706db1456931d35a: start","job_status":"start","queue_duration_s":0.269106,"scheduling_latency_s":0.269121}1298gitlab # [ 212.329859] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f3c4d02706db1456931d35a","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f3c4d02706db1456931d35a: done: 0.015402 sec","job_status":"done","queue_duration_s":0.269106,"scheduling_latency_s":0.269121,"redis_calls":13,"redis_duration_s":0.002065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001336,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015402,"completed_at":"2026-11-02T13:13:40.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1299gitlab # [ 212.370087] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26c73fdf7b46fc4f61abd7e7","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26c73fdf7b46fc4f61abd7e7: start","job_status":"start","queue_duration_s":0.286317,"scheduling_latency_s":0.286334}1300gitlab # [ 212.383024] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26c73fdf7b46fc4f61abd7e7","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26c73fdf7b46fc4f61abd7e7: done: 0.015478 sec","job_status":"done","queue_duration_s":0.286317,"scheduling_latency_s":0.286334,"redis_calls":13,"redis_duration_s":0.001974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015478,"completed_at":"2026-11-02T13:13:40.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1301gitlab # [ 212.423503] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"037dbdcade2f79aa7e4481fc","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-037dbdcade2f79aa7e4481fc: start","job_status":"start","queue_duration_s":0.302913,"scheduling_latency_s":0.302929}1302gitlab # [ 212.436536] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"037dbdcade2f79aa7e4481fc","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-037dbdcade2f79aa7e4481fc: done: 0.014903 sec","job_status":"done","queue_duration_s":0.302913,"scheduling_latency_s":0.302929,"redis_calls":13,"redis_duration_s":0.001872,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014903,"completed_at":"2026-11-02T13:13:40.799Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 212.477607] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e054baf31ada4763d1e79ffe","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e054baf31ada4763d1e79ffe: start","job_status":"start","queue_duration_s":0.318942,"scheduling_latency_s":0.318957}1304gitlab # [ 212.490178] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e054baf31ada4763d1e79ffe","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e054baf31ada4763d1e79ffe: done: 0.015371 sec","job_status":"done","queue_duration_s":0.318942,"scheduling_latency_s":0.318957,"redis_calls":13,"redis_duration_s":0.002132,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015371,"completed_at":"2026-11-02T13:13:40.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 212.530281] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfcd48eca7ca0b670799103b","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfcd48eca7ca0b670799103b: start","job_status":"start","queue_duration_s":0.335546,"scheduling_latency_s":0.335563}1306gitlab # [ 212.543120] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfcd48eca7ca0b670799103b","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfcd48eca7ca0b670799103b: done: 0.014719 sec","job_status":"done","queue_duration_s":0.335546,"scheduling_latency_s":0.335563,"redis_calls":13,"redis_duration_s":0.001856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014719,"completed_at":"2026-11-02T13:13:40.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 212.583761] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b517997ad76fd6e79ed64665","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b517997ad76fd6e79ed64665: start","job_status":"start","queue_duration_s":0.352492,"scheduling_latency_s":0.352508}1308gitlab # [ 212.596679] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b517997ad76fd6e79ed64665","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b517997ad76fd6e79ed64665: done: 0.016332 sec","job_status":"done","queue_duration_s":0.352492,"scheduling_latency_s":0.352508,"redis_calls":13,"redis_duration_s":0.002535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001862,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016332,"completed_at":"2026-11-02T13:13:40.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 212.637163] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dc0fd8b4abca9f643a37fed","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dc0fd8b4abca9f643a37fed: start","job_status":"start","queue_duration_s":0.369989,"scheduling_latency_s":0.370004}1310gitlab # [ 212.650931] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dc0fd8b4abca9f643a37fed","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dc0fd8b4abca9f643a37fed: done: 0.01597 sec","job_status":"done","queue_duration_s":0.369989,"scheduling_latency_s":0.370004,"redis_calls":13,"redis_duration_s":0.0021130000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01597,"completed_at":"2026-11-02T13:13:40.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 212.691478] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f02ec5a57a7fcf0481a7957","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f02ec5a57a7fcf0481a7957: start","job_status":"start","queue_duration_s":0.387245,"scheduling_latency_s":0.387261}1312gitlab # [ 212.704413] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f02ec5a57a7fcf0481a7957","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f02ec5a57a7fcf0481a7957: done: 0.016096 sec","job_status":"done","queue_duration_s":0.387245,"scheduling_latency_s":0.387261,"redis_calls":13,"redis_duration_s":0.002273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001321,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016096,"completed_at":"2026-11-02T13:13:40.885Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 212.744653] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9d6c365a4edc5640e4ddcd9","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9d6c365a4edc5640e4ddcd9: start","job_status":"start","queue_duration_s":0.405757,"scheduling_latency_s":0.405774}1314gitlab # [ 212.757480] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9d6c365a4edc5640e4ddcd9","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9d6c365a4edc5640e4ddcd9: done: 0.02457 sec","job_status":"done","queue_duration_s":0.405757,"scheduling_latency_s":0.405774,"redis_calls":13,"redis_duration_s":0.010058999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005134,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02457,"completed_at":"2026-11-02T13:13:40.912Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 212.797714] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85ff0de5602d3313d8d6f227","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85ff0de5602d3313d8d6f227: start","job_status":"start","queue_duration_s":0.43152,"scheduling_latency_s":0.431538}1316gitlab # [ 212.810384] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85ff0de5602d3313d8d6f227","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85ff0de5602d3313d8d6f227: done: 0.015231 sec","job_status":"done","queue_duration_s":0.43152,"scheduling_latency_s":0.431538,"redis_calls":13,"redis_duration_s":0.001705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015231,"completed_at":"2026-11-02T13:13:40.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 212.850771] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2c58887929ea9982e7204a4","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2c58887929ea9982e7204a4: start","job_status":"start","queue_duration_s":0.447985,"scheduling_latency_s":0.448002}1318gitlab # [ 212.864371] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2c58887929ea9982e7204a4","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2c58887929ea9982e7204a4: done: 0.015364 sec","job_status":"done","queue_duration_s":0.447985,"scheduling_latency_s":0.448002,"redis_calls":13,"redis_duration_s":0.001864,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001302,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015364,"completed_at":"2026-11-02T13:13:40.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 212.904699] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cdc1f2d315260f942185e82","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cdc1f2d315260f942185e82: start","job_status":"start","queue_duration_s":0.464525,"scheduling_latency_s":0.464542}1320gitlab # [ 212.917920] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cdc1f2d315260f942185e82","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cdc1f2d315260f942185e82: done: 0.015099 sec","job_status":"done","queue_duration_s":0.464525,"scheduling_latency_s":0.464542,"redis_calls":13,"redis_duration_s":0.001446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015099,"completed_at":"2026-11-02T13:13:40.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 212.958212] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13581d7248dcb5164e8b00be","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13581d7248dcb5164e8b00be: start","job_status":"start","queue_duration_s":0.480716,"scheduling_latency_s":0.480733}1322gitlab # [ 212.971127] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13581d7248dcb5164e8b00be","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13581d7248dcb5164e8b00be: done: 0.014513 sec","job_status":"done","queue_duration_s":0.480716,"scheduling_latency_s":0.480733,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014513,"completed_at":"2026-11-02T13:13:40.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 213.011400] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15fef55ebc85b13dbb5838da","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15fef55ebc85b13dbb5838da: start","job_status":"start","queue_duration_s":0.496325,"scheduling_latency_s":0.496342}1324gitlab # [ 213.024380] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15fef55ebc85b13dbb5838da","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15fef55ebc85b13dbb5838da: done: 0.014562 sec","job_status":"done","queue_duration_s":0.496325,"scheduling_latency_s":0.496342,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014562,"completed_at":"2026-11-02T13:13:40.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 213.065523] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:40.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"836dbb4eaa730c20e6674741","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-836dbb4eaa730c20e6674741: start","job_status":"start","queue_duration_s":0.512085,"scheduling_latency_s":0.512133}1326gitlab # [ 213.078228] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"836dbb4eaa730c20e6674741","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-836dbb4eaa730c20e6674741: done: 0.014528 sec","job_status":"done","queue_duration_s":0.512085,"scheduling_latency_s":0.512133,"redis_calls":13,"redis_duration_s":0.001411,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014528,"completed_at":"2026-11-02T13:13:41.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 213.118486] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5e9783fb2820c84e2259420","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5e9783fb2820c84e2259420: start","job_status":"start","queue_duration_s":0.528048,"scheduling_latency_s":0.528065}1328gitlab # [ 213.131371] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5e9783fb2820c84e2259420","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5e9783fb2820c84e2259420: done: 0.014567 sec","job_status":"done","queue_duration_s":0.528048,"scheduling_latency_s":0.528065,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014567,"completed_at":"2026-11-02T13:13:41.024Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 213.171780] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc59cce4eda517aaeaa27d2f","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc59cce4eda517aaeaa27d2f: start","job_status":"start","queue_duration_s":0.543807,"scheduling_latency_s":0.543826}1330gitlab # [ 213.184675] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc59cce4eda517aaeaa27d2f","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc59cce4eda517aaeaa27d2f: done: 0.014813 sec","job_status":"done","queue_duration_s":0.543807,"scheduling_latency_s":0.543826,"redis_calls":13,"redis_duration_s":0.001469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014813,"completed_at":"2026-11-02T13:13:41.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 213.224866] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3d2a246e98e955ec5a859be","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3d2a246e98e955ec5a859be: start","job_status":"start","queue_duration_s":0.559817,"scheduling_latency_s":0.559834}1332gitlab # [ 213.237388] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3d2a246e98e955ec5a859be","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3d2a246e98e955ec5a859be: done: 0.014745 sec","job_status":"done","queue_duration_s":0.559817,"scheduling_latency_s":0.559834,"redis_calls":13,"redis_duration_s":0.0014559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014745,"completed_at":"2026-11-02T13:13:41.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 213.278404] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a272204a360d95a62b663f6","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a272204a360d95a62b663f6: start","job_status":"start","queue_duration_s":0.575912,"scheduling_latency_s":0.575929}1334gitlab # [ 213.291223] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a272204a360d95a62b663f6","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a272204a360d95a62b663f6: done: 0.014571 sec","job_status":"done","queue_duration_s":0.575912,"scheduling_latency_s":0.575929,"redis_calls":13,"redis_duration_s":0.0017950000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014571,"completed_at":"2026-11-02T13:13:41.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 213.331634] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30fd19832d1a885ac1197037","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30fd19832d1a885ac1197037: start","job_status":"start","queue_duration_s":0.59158,"scheduling_latency_s":0.591597}1336gitlab # [ 213.344412] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30fd19832d1a885ac1197037","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30fd19832d1a885ac1197037: done: 0.014457 sec","job_status":"done","queue_duration_s":0.59158,"scheduling_latency_s":0.591597,"redis_calls":13,"redis_duration_s":0.0014299999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014457,"completed_at":"2026-11-02T13:13:41.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 213.384666] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5edd3117e241d0f60a720d37","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5edd3117e241d0f60a720d37: start","job_status":"start","queue_duration_s":0.607106,"scheduling_latency_s":0.607123}1338gitlab # [ 213.397791] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5edd3117e241d0f60a720d37","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5edd3117e241d0f60a720d37: done: 0.01437 sec","job_status":"done","queue_duration_s":0.607106,"scheduling_latency_s":0.607123,"redis_calls":13,"redis_duration_s":0.001439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01437,"completed_at":"2026-11-02T13:13:41.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 213.437460] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be26f81c37075ba7536c910a","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be26f81c37075ba7536c910a: start","job_status":"start","queue_duration_s":0.622555,"scheduling_latency_s":0.622571}1340gitlab # [ 213.450173] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be26f81c37075ba7536c910a","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be26f81c37075ba7536c910a: done: 0.014737 sec","job_status":"done","queue_duration_s":0.622555,"scheduling_latency_s":0.622571,"redis_calls":13,"redis_duration_s":0.0014060000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014737,"completed_at":"2026-11-02T13:13:41.119Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 213.491188] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7ff644e91d9b9955ee768ab","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7ff644e91d9b9955ee768ab: start","job_status":"start","queue_duration_s":0.638407,"scheduling_latency_s":0.638424}1342gitlab # [ 213.504045] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7ff644e91d9b9955ee768ab","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7ff644e91d9b9955ee768ab: done: 0.01453 sec","job_status":"done","queue_duration_s":0.638407,"scheduling_latency_s":0.638424,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01453,"completed_at":"2026-11-02T13:13:41.134Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 213.543847] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de8896b6e11a891be40f5370","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de8896b6e11a891be40f5370: start","job_status":"start","queue_duration_s":0.654078,"scheduling_latency_s":0.654095}1344gitlab # [ 213.556651] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de8896b6e11a891be40f5370","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de8896b6e11a891be40f5370: done: 0.014936 sec","job_status":"done","queue_duration_s":0.654078,"scheduling_latency_s":0.654095,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014936,"completed_at":"2026-11-02T13:13:41.150Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 213.597195] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a624a131b7e55306ec3b73","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a624a131b7e55306ec3b73: start","job_status":"start","queue_duration_s":0.670183,"scheduling_latency_s":0.670199}1346gitlab # [ 213.609789] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a624a131b7e55306ec3b73","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a624a131b7e55306ec3b73: done: 0.015042 sec","job_status":"done","queue_duration_s":0.670183,"scheduling_latency_s":0.670199,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015042,"completed_at":"2026-11-02T13:13:41.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 213.649846] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3da221ef03b2fa74b4dba91","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3da221ef03b2fa74b4dba91: start","job_status":"start","queue_duration_s":0.686445,"scheduling_latency_s":0.686462}1348gitlab # [ 213.662594] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3da221ef03b2fa74b4dba91","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3da221ef03b2fa74b4dba91: done: 0.014677 sec","job_status":"done","queue_duration_s":0.686445,"scheduling_latency_s":0.686462,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014677,"completed_at":"2026-11-02T13:13:41.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 213.703783] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85dfb6d80872b0c98023e2c3","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85dfb6d80872b0c98023e2c3: start","job_status":"start","queue_duration_s":0.702289,"scheduling_latency_s":0.702305}1350gitlab # [ 213.716782] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85dfb6d80872b0c98023e2c3","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85dfb6d80872b0c98023e2c3: done: 0.014654 sec","job_status":"done","queue_duration_s":0.702289,"scheduling_latency_s":0.702305,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014654,"completed_at":"2026-11-02T13:13:41.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 213.756945] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a27eb6dcd595f817e4cdbd49","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a27eb6dcd595f817e4cdbd49: start","job_status":"start","queue_duration_s":0.7181,"scheduling_latency_s":0.718117}1352gitlab # [ 213.769621] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a27eb6dcd595f817e4cdbd49","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a27eb6dcd595f817e4cdbd49: done: 0.014404 sec","job_status":"done","queue_duration_s":0.7181,"scheduling_latency_s":0.718117,"redis_calls":13,"redis_duration_s":0.001402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014404,"completed_at":"2026-11-02T13:13:41.214Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 213.809590] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"639f0fe87d33c09860f69b8e","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-639f0fe87d33c09860f69b8e: start","job_status":"start","queue_duration_s":0.733797,"scheduling_latency_s":0.733813}1354gitlab # [ 213.822385] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"639f0fe87d33c09860f69b8e","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-639f0fe87d33c09860f69b8e: done: 0.015913 sec","job_status":"done","queue_duration_s":0.733797,"scheduling_latency_s":0.733813,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015913,"completed_at":"2026-11-02T13:13:41.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 213.862520] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f89940b6e0a3e8de0ef290a3","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f89940b6e0a3e8de0ef290a3: start","job_status":"start","queue_duration_s":0.750816,"scheduling_latency_s":0.750833}1356gitlab # [ 213.876342] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f89940b6e0a3e8de0ef290a3","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f89940b6e0a3e8de0ef290a3: done: 0.015196 sec","job_status":"done","queue_duration_s":0.750816,"scheduling_latency_s":0.750833,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015196,"completed_at":"2026-11-02T13:13:41.247Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 213.916571] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e337d58daaad7e3a3955e33b","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e337d58daaad7e3a3955e33b: start","job_status":"start","queue_duration_s":0.767151,"scheduling_latency_s":0.767169}1358gitlab # [ 213.929144] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e337d58daaad7e3a3955e33b","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e337d58daaad7e3a3955e33b: done: 0.014543 sec","job_status":"done","queue_duration_s":0.767151,"scheduling_latency_s":0.767169,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014543,"completed_at":"2026-11-02T13:13:41.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 213.969409] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a3a2a978e4cc2b1e3c57ea7","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a3a2a978e4cc2b1e3c57ea7: start","job_status":"start","queue_duration_s":0.782988,"scheduling_latency_s":0.783007}1360gitlab # [ 213.982047] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a3a2a978e4cc2b1e3c57ea7","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a3a2a978e4cc2b1e3c57ea7: done: 0.014614 sec","job_status":"done","queue_duration_s":0.782988,"scheduling_latency_s":0.783007,"redis_calls":13,"redis_duration_s":0.0014420000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014614,"completed_at":"2026-11-02T13:13:41.279Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 214.022410] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7ac4720932ccd8f4aceac1","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7ac4720932ccd8f4aceac1: start","job_status":"start","queue_duration_s":0.798511,"scheduling_latency_s":0.798528}1362gitlab # [ 214.035139] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.295Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7ac4720932ccd8f4aceac1","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7ac4720932ccd8f4aceac1: done: 0.014756 sec","job_status":"done","queue_duration_s":0.798511,"scheduling_latency_s":0.798528,"redis_calls":13,"redis_duration_s":0.0014420000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014756,"completed_at":"2026-11-02T13:13:41.295Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 214.075477] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a7aeeb18b26bb69015ff35","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a7aeeb18b26bb69015ff35: start","job_status":"start","queue_duration_s":0.814378,"scheduling_latency_s":0.814395}1364gitlab # [ 214.089344] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a7aeeb18b26bb69015ff35","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a7aeeb18b26bb69015ff35: done: 0.014615 sec","job_status":"done","queue_duration_s":0.814378,"scheduling_latency_s":0.814395,"redis_calls":13,"redis_duration_s":0.0018210000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014615,"completed_at":"2026-11-02T13:13:41.310Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 214.129443] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22ea63796b84aaef60c5db8d","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22ea63796b84aaef60c5db8d: start","job_status":"start","queue_duration_s":0.830087,"scheduling_latency_s":0.830103}1366gitlab # [ 214.142159] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22ea63796b84aaef60c5db8d","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22ea63796b84aaef60c5db8d: done: 0.015275 sec","job_status":"done","queue_duration_s":0.830087,"scheduling_latency_s":0.830103,"redis_calls":13,"redis_duration_s":0.001502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015275,"completed_at":"2026-11-02T13:13:41.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 214.182142] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2516b2df7e965dda7cb99444","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2516b2df7e965dda7cb99444: start","job_status":"start","queue_duration_s":0.846472,"scheduling_latency_s":0.84649}1368gitlab # [ 214.194972] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2516b2df7e965dda7cb99444","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2516b2df7e965dda7cb99444: done: 0.015614 sec","job_status":"done","queue_duration_s":0.846472,"scheduling_latency_s":0.84649,"redis_calls":13,"redis_duration_s":0.001663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015614,"completed_at":"2026-11-02T13:13:41.343Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 214.234980] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3792484cfc84325e5ed7490c","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3792484cfc84325e5ed7490c: start","job_status":"start","queue_duration_s":0.863175,"scheduling_latency_s":0.863192}1370gitlab # [ 214.247629] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3792484cfc84325e5ed7490c","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3792484cfc84325e5ed7490c: done: 0.016035 sec","job_status":"done","queue_duration_s":0.863175,"scheduling_latency_s":0.863192,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016035,"completed_at":"2026-11-02T13:13:41.361Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 214.288915] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03b810acde1eac9f9bcf281f","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03b810acde1eac9f9bcf281f: start","job_status":"start","queue_duration_s":0.880318,"scheduling_latency_s":0.880335}1372gitlab # [ 214.301450] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03b810acde1eac9f9bcf281f","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03b810acde1eac9f9bcf281f: done: 0.0163 sec","job_status":"done","queue_duration_s":0.880318,"scheduling_latency_s":0.880335,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0163,"completed_at":"2026-11-02T13:13:41.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 214.341487] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"506590e1352837091cecae62","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-506590e1352837091cecae62: start","job_status":"start","queue_duration_s":0.897682,"scheduling_latency_s":0.897698}1374gitlab # [ 214.354318] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"506590e1352837091cecae62","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-506590e1352837091cecae62: done: 0.016295 sec","job_status":"done","queue_duration_s":0.897682,"scheduling_latency_s":0.897698,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016295,"completed_at":"2026-11-02T13:13:41.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 214.394723] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7d861141922737f607852f7","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7d861141922737f607852f7: start","job_status":"start","queue_duration_s":0.915086,"scheduling_latency_s":0.915103}1376gitlab # [ 214.407401] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7d861141922737f607852f7","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7d861141922737f607852f7: done: 0.016163 sec","job_status":"done","queue_duration_s":0.915086,"scheduling_latency_s":0.915103,"redis_calls":13,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016163,"completed_at":"2026-11-02T13:13:41.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 214.447870] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0160c79587ec785b76aef4bb","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0160c79587ec785b76aef4bb: start","job_status":"start","queue_duration_s":0.932354,"scheduling_latency_s":0.93237}1378gitlab # [ 214.460916] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0160c79587ec785b76aef4bb","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0160c79587ec785b76aef4bb: done: 0.015668 sec","job_status":"done","queue_duration_s":0.932354,"scheduling_latency_s":0.93237,"redis_calls":13,"redis_duration_s":0.0014759999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015668,"completed_at":"2026-11-02T13:13:41.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 214.502040] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b9710075449d868aa0c6ed","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b9710075449d868aa0c6ed: start","job_status":"start","queue_duration_s":0.949454,"scheduling_latency_s":0.949472}1380gitlab # [ 214.514672] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:13:41.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b9710075449d868aa0c6ed","created_at":"2026-11-02T13:13:40.293Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-e53ab42c33444538","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"d2b5321f305d825639616d6b53d50540","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:13:40.481Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b9710075449d868aa0c6ed: done: 0.016223 sec","job_status":"done","queue_duration_s":0.949454,"scheduling_latency_s":0.949472,"redis_calls":13,"redis_duration_s":0.0017429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016223,"completed_at":"2026-11-02T13:13:41.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 233.536467] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:02.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"78c6b27f90d540a29d45b001","created_at":"2026-11-02T13:14:02.500Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d43c1275851be7ab641aab43b08b8e11","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.514Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-78c6b27f90d540a29d45b001: start","job_status":"start","queue_duration_s":0.002351,"scheduling_latency_s":0.002389}1382gitlab # [ 233.827216] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:02.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"78c6b27f90d540a29d45b001","created_at":"2026-11-02T13:14:02.500Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d43c1275851be7ab641aab43b08b8e11","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.514Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-78c6b27f90d540a29d45b001: done: 0.291009 sec","job_status":"done","queue_duration_s":0.002351,"scheduling_latency_s":0.002389,"redis_calls":12,"redis_duration_s":0.013057000000000001,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002618,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005817,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.199166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00239668,"exclusive_lock_hold_duration_s":0.2423895,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.291009,"completed_at":"2026-11-02T13:14:02.808Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050455,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1383gitlab # [ 233.900028] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:02.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"cb255a13a7ad6a7b7f0d59ed","created_at":"2026-11-02T13:14:02.525Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bade80be04a74da704df5af457cc47f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.542Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-cb255a13a7ad6a7b7f0d59ed: start","job_status":"start","queue_duration_s":0.307155,"scheduling_latency_s":0.307187}1384gitlab # [ 233.943007] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:02.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"cb255a13a7ad6a7b7f0d59ed","created_at":"2026-11-02T13:14:02.525Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bade80be04a74da704df5af457cc47f5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.542Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-cb255a13a7ad6a7b7f0d59ed: done: 0.074151 sec","job_status":"done","queue_duration_s":0.307155,"scheduling_latency_s":0.307187,"redis_calls":7,"redis_duration_s":0.044978,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.041921,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003057,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.074151,"completed_at":"2026-11-02T13:14:02.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 234.000085] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:02.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0c76f1186540aeaf88be7d6c","created_at":"2026-11-02T13:14:02.562Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"001f5e547c8f582f0c78e0123a3c1f9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.567Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0c76f1186540aeaf88be7d6c: start","job_status":"start","queue_duration_s":0.359527,"scheduling_latency_s":0.359567}1386gitlab # [ 234.468654] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0c76f1186540aeaf88be7d6c","created_at":"2026-11-02T13:14:02.562Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"001f5e547c8f582f0c78e0123a3c1f9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.567Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0c76f1186540aeaf88be7d6c: done: 0.523314 sec","job_status":"done","queue_duration_s":0.359527,"scheduling_latency_s":0.359567,"redis_calls":11,"redis_duration_s":0.002833,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.114,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.114,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.256724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00121704,"exclusive_lock_hold_duration_s":0.49665748,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.523314,"completed_at":"2026-11-02T13:14:03.449Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.284432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 234.540510] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.455Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8e3279044a278160afe5ce22","created_at":"2026-11-02T13:14:02.583Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d8a58163df8a04aace6787ac43216f1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.587Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8e3279044a278160afe5ce22: start","job_status":"start","queue_duration_s":0.868155,"scheduling_latency_s":0.868178}1388gitlab # [ 234.560066] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.471Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"8e3279044a278160afe5ce22","created_at":"2026-11-02T13:14:02.583Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d8a58163df8a04aace6787ac43216f1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.587Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-8e3279044a278160afe5ce22: done: 0.016268 sec","job_status":"done","queue_duration_s":0.868155,"scheduling_latency_s":0.868178,"redis_calls":7,"redis_duration_s":0.001268,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016268,"completed_at":"2026-11-02T13:14:03.471Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 234.605538] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.472Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"521273351e7a0d7aea8d8cb7","created_at":"2026-11-02T13:14:02.609Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"575efa1189a8e8fae85f115a7fb59274","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-02T13:14:02.617Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-521273351e7a0d7aea8d8cb7: start","job_status":"start","queue_duration_s":0.855814,"scheduling_latency_s":0.855832}1390gitlab # [ 234.619304] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.483Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1391gitlab # [ 234.623232] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.489Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"521273351e7a0d7aea8d8cb7","created_at":"2026-11-02T13:14:02.609Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"575efa1189a8e8fae85f115a7fb59274","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-02T13:14:02.617Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-521273351e7a0d7aea8d8cb7: done: 0.016786 sec","job_status":"done","queue_duration_s":0.855814,"scheduling_latency_s":0.855832,"redis_calls":7,"redis_duration_s":0.004399,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016786,"completed_at":"2026-11-02T13:14:03.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 234.663750] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1ae94f8824e69f55d2793a93","created_at":"2026-11-02T13:14:02.792Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8bdaae0e54dd678391ea2f39ca6b898","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.799Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1ae94f8824e69f55d2793a93: start","job_status":"start","queue_duration_s":0.692853,"scheduling_latency_s":0.692873}1393gitlab # [ 234.678925] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1ae94f8824e69f55d2793a93","created_at":"2026-11-02T13:14:02.792Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8bdaae0e54dd678391ea2f39ca6b898","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.799Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1ae94f8824e69f55d2793a93: done: 0.025788 sec","job_status":"done","queue_duration_s":0.692853,"scheduling_latency_s":0.692873,"redis_calls":10,"redis_duration_s":0.001564,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000701,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00567284,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.025788,"completed_at":"2026-11-02T13:14:03.518Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002859,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1394gitlab # [ 234.724074] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3f5e0f0bfe87d690d67f7ea1","created_at":"2026-11-02T13:14:02.847Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63ee49f8d50f8318e71e7c28a64f1535","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-02T13:14:02.852Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3f5e0f0bfe87d690d67f7ea1: start","job_status":"start","queue_duration_s":0.66755,"scheduling_latency_s":0.667568}1395gitlab # [ 234.737779] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.527Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1396gitlab # [ 234.741246] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3f5e0f0bfe87d690d67f7ea1","created_at":"2026-11-02T13:14:02.847Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63ee49f8d50f8318e71e7c28a64f1535","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-02T13:14:02.852Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3f5e0f0bfe87d690d67f7ea1: done: 0.012767 sec","job_status":"done","queue_duration_s":0.66755,"scheduling_latency_s":0.667568,"redis_calls":7,"redis_duration_s":0.000973,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012767,"completed_at":"2026-11-02T13:14:03.532Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 234.781463] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"911fb67f09401566c8869116","created_at":"2026-11-02T13:14:02.910Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b99cb4702efcddf99ff5c9bcc15dbad3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.958Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-911fb67f09401566c8869116: start","job_status":"start","queue_duration_s":0.575308,"scheduling_latency_s":0.575325}1398gitlab # [ 234.796449] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"911fb67f09401566c8869116","created_at":"2026-11-02T13:14:02.910Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b99cb4702efcddf99ff5c9bcc15dbad3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.958Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-911fb67f09401566c8869116: done: 0.027305 sec","job_status":"done","queue_duration_s":0.575308,"scheduling_latency_s":0.575325,"redis_calls":10,"redis_duration_s":0.001567,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025414,"duration_s":0.027305,"completed_at":"2026-11-02T13:14:03.560Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 234.840033] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6248d9d40e61ec7590816697","created_at":"2026-11-02T13:14:02.969Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"117d7011cb0b9f9e76f99e2d82feb3c0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.980Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6248d9d40e61ec7590816697: start","job_status":"start","queue_duration_s":0.58156,"scheduling_latency_s":0.581578}1400gitlab # [ 234.854554] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6248d9d40e61ec7590816697","created_at":"2026-11-02T13:14:02.969Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"117d7011cb0b9f9e76f99e2d82feb3c0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:02.980Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6248d9d40e61ec7590816697: done: 0.026924 sec","job_status":"done","queue_duration_s":0.58156,"scheduling_latency_s":0.581578,"redis_calls":12,"redis_duration_s":0.001859,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048762,"exclusive_lock_hold_duration_s":0.0055092399999999995,"duration_s":0.026924,"completed_at":"2026-11-02T13:14:03.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 234.898285] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b49d1f7c467d146fd2680199","created_at":"2026-11-02T13:14:03.013Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c7d24b1be0572af81212571870787fe","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.016Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b49d1f7c467d146fd2680199: start","job_status":"start","queue_duration_s":0.574047,"scheduling_latency_s":0.574063}1402gitlab # [ 234.914272] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b49d1f7c467d146fd2680199","created_at":"2026-11-02T13:14:03.013Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c7d24b1be0572af81212571870787fe","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.016Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b49d1f7c467d146fd2680199: done: 0.016105 sec","job_status":"done","queue_duration_s":0.574047,"scheduling_latency_s":0.574063,"redis_calls":9,"redis_duration_s":0.001388,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000664,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048026,"exclusive_lock_hold_duration_s":0.0029532,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.016105,"completed_at":"2026-11-02T13:14:03.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 234.956949] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a0d80607a0b25cb5b13e5f3c","created_at":"2026-11-02T13:14:03.023Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccdcda35daa71ca7d356374bae8f92c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.031Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a0d80607a0b25cb5b13e5f3c: start","job_status":"start","queue_duration_s":0.576065,"scheduling_latency_s":0.576082}1404gitlab # [ 234.971764] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a0d80607a0b25cb5b13e5f3c","created_at":"2026-11-02T13:14:03.023Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccdcda35daa71ca7d356374bae8f92c6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.031Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a0d80607a0b25cb5b13e5f3c: done: 0.041266 sec","job_status":"done","queue_duration_s":0.576065,"scheduling_latency_s":0.576082,"redis_calls":9,"redis_duration_s":0.001899,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00064812,"exclusive_lock_hold_duration_s":0.01617848,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.041266,"completed_at":"2026-11-02T13:14:03.649Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 235.016437] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"33b1113652cc94e6e6d2eabe","created_at":"2026-11-02T13:14:03.045Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.050Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-33b1113652cc94e6e6d2eabe: start","job_status":"start","queue_duration_s":0.600382,"scheduling_latency_s":0.6004}1406gitlab # [ 235.030055] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"33b1113652cc94e6e6d2eabe","created_at":"2026-11-02T13:14:03.045Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.050Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-33b1113652cc94e6e6d2eabe: done: 0.027965 sec","job_status":"done","queue_duration_s":0.600382,"scheduling_latency_s":0.6004,"redis_calls":14,"redis_duration_s":0.0030689999999999997,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000256,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002116,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000697,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027965,"completed_at":"2026-11-02T13:14:03.678Z","load_balancing_strategy":"primary","db_duration_s":0.001362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 235.072119] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1b179a8b9042e01ed155353a","created_at":"2026-11-02T13:14:03.079Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9f776eaacab12b4956963bc2bfc4f84","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.084Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1b179a8b9042e01ed155353a: start","job_status":"start","queue_duration_s":0.595351,"scheduling_latency_s":0.59537}1408gitlab # [ 235.086730] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.721Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"1b179a8b9042e01ed155353a","created_at":"2026-11-02T13:14:03.079Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9f776eaacab12b4956963bc2bfc4f84","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.084Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-1b179a8b9042e01ed155353a: done: 0.041999 sec","job_status":"done","queue_duration_s":0.595351,"scheduling_latency_s":0.59537,"redis_calls":9,"redis_duration_s":0.0017339999999999999,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000843,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000479,"exclusive_lock_hold_duration_s":0.02769868,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.041999,"completed_at":"2026-11-02T13:14:03.721Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 235.132342] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.723Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d5a9e07a9c1e6ab1fc75597d","created_at":"2026-11-02T13:14:03.098Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34df19083f57aab203ae3d97cdca7373","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.102Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d5a9e07a9c1e6ab1fc75597d: start","job_status":"start","queue_duration_s":0.621354,"scheduling_latency_s":0.621372}1410gitlab # [ 235.146331] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d5a9e07a9c1e6ab1fc75597d","created_at":"2026-11-02T13:14:03.098Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34df19083f57aab203ae3d97cdca7373","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.102Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d5a9e07a9c1e6ab1fc75597d: done: 0.024807 sec","job_status":"done","queue_duration_s":0.621354,"scheduling_latency_s":0.621372,"redis_calls":7,"redis_duration_s":0.001507,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.024807,"completed_at":"2026-11-02T13:14:03.748Z","load_balancing_strategy":"primary","db_duration_s":0.009417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 235.188437] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8dded08147f90f86d5baebf5","created_at":"2026-11-02T13:14:03.112Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa653d43638fad2e444024ee6fc8c71","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.115Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8dded08147f90f86d5baebf5: start","job_status":"start","queue_duration_s":0.633664,"scheduling_latency_s":0.633683}1412gitlab # [ 235.203232] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.758Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1413gitlab # [ 235.206759] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8dded08147f90f86d5baebf5","created_at":"2026-11-02T13:14:03.112Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baa653d43638fad2e444024ee6fc8c71","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.115Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8dded08147f90f86d5baebf5: done: 0.013932 sec","job_status":"done","queue_duration_s":0.633664,"scheduling_latency_s":0.633683,"redis_calls":7,"redis_duration_s":0.001531,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013932,"completed_at":"2026-11-02T13:14:03.763Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 235.248721] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7fcb12f93c57030a4b0a2c08","created_at":"2026-11-02T13:14:03.131Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af7a185f322b74a69a67614aab6aad27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.139Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7fcb12f93c57030a4b0a2c08: start","job_status":"start","queue_duration_s":0.625384,"scheduling_latency_s":0.625401}1415gitlab # [ 235.263351] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7fcb12f93c57030a4b0a2c08","created_at":"2026-11-02T13:14:03.131Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af7a185f322b74a69a67614aab6aad27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.139Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7fcb12f93c57030a4b0a2c08: done: 0.013262 sec","job_status":"done","queue_duration_s":0.625384,"scheduling_latency_s":0.625401,"redis_calls":7,"redis_duration_s":0.001185,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013262,"completed_at":"2026-11-02T13:14:03.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 235.305138] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:03.779Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bceda37ad9713a8d269fe3cb","created_at":"2026-11-02T13:14:03.154Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efed9948c78000010d7be747743450b5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.158Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bceda37ad9713a8d269fe3cb: start","job_status":"start","queue_duration_s":0.620959,"scheduling_latency_s":0.620973}1417gitlab # [ 235.320625] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bceda37ad9713a8d269fe3cb","created_at":"2026-11-02T13:14:03.154Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efed9948c78000010d7be747743450b5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.158Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bceda37ad9713a8d269fe3cb: done: 0.357505 sec","job_status":"done","queue_duration_s":0.620959,"scheduling_latency_s":0.620973,"redis_calls":562,"redis_duration_s":0.053463,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.052543,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.336584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.357505,"completed_at":"2026-11-02T13:14:04.136Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 235.361966] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3a48dd03da9d2fd51566394b","created_at":"2026-11-02T13:14:03.170Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fcb42a6298fa8e410e86cfdf49c4558","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.174Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3a48dd03da9d2fd51566394b: start","job_status":"start","queue_duration_s":0.963993,"scheduling_latency_s":0.964012}1419gitlab # [ 235.376950] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.145Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1420gitlab # [ 235.380821] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3a48dd03da9d2fd51566394b","created_at":"2026-11-02T13:14:03.170Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fcb42a6298fa8e410e86cfdf49c4558","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.174Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3a48dd03da9d2fd51566394b: done: 0.012537 sec","job_status":"done","queue_duration_s":0.963993,"scheduling_latency_s":0.964012,"redis_calls":7,"redis_duration_s":0.001147,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012537,"completed_at":"2026-11-02T13:14:04.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 235.422561] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3e2977f511f5d6b11c6c53df","created_at":"2026-11-02T13:14:03.198Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85d8c0cccb2fc45ae5da34c84cb74fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.206Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3e2977f511f5d6b11c6c53df: start","job_status":"start","queue_duration_s":0.945025,"scheduling_latency_s":0.945041}1422gitlab # [ 235.437818] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.159Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1423gitlab # [ 235.441151] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3e2977f511f5d6b11c6c53df","created_at":"2026-11-02T13:14:03.198Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85d8c0cccb2fc45ae5da34c84cb74fb2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.206Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3e2977f511f5d6b11c6c53df: done: 0.012396 sec","job_status":"done","queue_duration_s":0.945025,"scheduling_latency_s":0.945041,"redis_calls":7,"redis_duration_s":0.001392,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012396,"completed_at":"2026-11-02T13:14:04.164Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 235.482431] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"feaa4bf1c411043dd32e9b7e","created_at":"2026-11-02T13:14:03.220Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47421f9836faee8149d5795ce7d2e581","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.223Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-feaa4bf1c411043dd32e9b7e: start","job_status":"start","queue_duration_s":0.941796,"scheduling_latency_s":0.941812}1425gitlab # [ 235.497131] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.173Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1426gitlab # [ 235.500811] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"feaa4bf1c411043dd32e9b7e","created_at":"2026-11-02T13:14:03.220Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47421f9836faee8149d5795ce7d2e581","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.223Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-feaa4bf1c411043dd32e9b7e: done: 0.013699 sec","job_status":"done","queue_duration_s":0.941796,"scheduling_latency_s":0.941812,"redis_calls":7,"redis_duration_s":0.0011099999999999999,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000307,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013699,"completed_at":"2026-11-02T13:14:04.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 235.543909] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d107fd99029a66257aeacb6c","created_at":"2026-11-02T13:14:03.240Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8cb7388488fb46ca0c9183446e99db12","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.247Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d107fd99029a66257aeacb6c: start","job_status":"start","queue_duration_s":0.932588,"scheduling_latency_s":0.932603}1428gitlab # [ 235.557719] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.199Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d107fd99029a66257aeacb6c","created_at":"2026-11-02T13:14:03.240Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8cb7388488fb46ca0c9183446e99db12","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.247Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d107fd99029a66257aeacb6c: done: 0.019592 sec","job_status":"done","queue_duration_s":0.932588,"scheduling_latency_s":0.932603,"redis_calls":12,"redis_duration_s":0.002076,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000447,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000375,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026452,"duration_s":0.019592,"completed_at":"2026-11-02T13:14:04.199Z","load_balancing_strategy":"primary","db_duration_s":0.000621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 235.601089] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ecd37408d7b811a01387feff","created_at":"2026-11-02T13:14:03.264Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc95b2e05196bee958260fcfc9312274","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-02T13:14:03.268Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ecd37408d7b811a01387feff: start","job_status":"start","queue_duration_s":0.934081,"scheduling_latency_s":0.934098}1430gitlab # [ 235.614836] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.223Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ecd37408d7b811a01387feff","created_at":"2026-11-02T13:14:03.264Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc95b2e05196bee958260fcfc9312274","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-02T13:14:03.268Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ecd37408d7b811a01387feff: done: 0.021613 sec","job_status":"done","queue_duration_s":0.934081,"scheduling_latency_s":0.934098,"redis_calls":7,"redis_duration_s":0.001318,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021613,"completed_at":"2026-11-02T13:14:04.223Z","load_balancing_strategy":"primary","db_duration_s":0.002144,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 235.655430] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"139cd2e4202944d3f8281a82","created_at":"2026-11-02T13:14:03.286Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38f81f81787d9c005509e44e9af7f469","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.290Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-139cd2e4202944d3f8281a82: start","job_status":"start","queue_duration_s":0.93477,"scheduling_latency_s":0.934786}1432gitlab # [ 235.670075] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"139cd2e4202944d3f8281a82","created_at":"2026-11-02T13:14:03.286Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38f81f81787d9c005509e44e9af7f469","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.290Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-139cd2e4202944d3f8281a82: done: 0.020058 sec","job_status":"done","queue_duration_s":0.93477,"scheduling_latency_s":0.934786,"redis_calls":7,"redis_duration_s":0.0012799999999999999,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00034,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020058,"completed_at":"2026-11-02T13:14:04.244Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003972,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 235.711396] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.247Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"222dd410c3de0c46ceae480a","created_at":"2026-11-02T13:14:03.301Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.308Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-222dd410c3de0c46ceae480a: start","job_status":"start","queue_duration_s":0.938461,"scheduling_latency_s":0.938478}1434gitlab # [ 235.726547] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"222dd410c3de0c46ceae480a","created_at":"2026-11-02T13:14:03.301Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.308Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-222dd410c3de0c46ceae480a: done: 0.093768 sec","job_status":"done","queue_duration_s":0.938461,"scheduling_latency_s":0.938478,"redis_calls":60,"redis_duration_s":0.013552000000000002,"redis_read_bytes":108,"redis_write_bytes":41009,"redis_queues_calls":2,"redis_queues_duration_s":0.00051,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.012573,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.093768,"completed_at":"2026-11-02T13:14:04.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 235.769031] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"89fecf448c9a77bb31748d2d","created_at":"2026-11-02T13:14:03.321Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee48fa75c3394e27241df142b94bc796","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.324Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-89fecf448c9a77bb31748d2d: start","job_status":"start","queue_duration_s":1.017175,"scheduling_latency_s":1.017192}1436gitlab # [ 235.782989] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.359Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"89fecf448c9a77bb31748d2d","created_at":"2026-11-02T13:14:03.321Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-aaf4906353da4727","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee48fa75c3394e27241df142b94bc796","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.324Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-89fecf448c9a77bb31748d2d: done: 0.017564 sec","job_status":"done","queue_duration_s":1.017175,"scheduling_latency_s":1.017192,"redis_calls":9,"redis_duration_s":0.001697,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00049634,"exclusive_lock_hold_duration_s":0.0048061,"duration_s":0.017564,"completed_at":"2026-11-02T13:14:04.359Z","load_balancing_strategy":"primary","db_duration_s":0.001375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 235.824058] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.361Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8640e22dfa2675999f2b0c48","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8640e22dfa2675999f2b0c48: start","job_status":"start","queue_duration_s":0.688656,"scheduling_latency_s":0.688673}1438gitlab # [ 235.836691] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.373Z","message":"No existing merge request to be cleaned up."}1439gitlab # [ 235.838411] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.381Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8640e22dfa2675999f2b0c48","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8640e22dfa2675999f2b0c48: done: 0.020155 sec","job_status":"done","queue_duration_s":0.688656,"scheduling_latency_s":0.688673,"redis_calls":10,"redis_duration_s":0.0017820000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020155,"completed_at":"2026-11-02T13:14:04.381Z","load_balancing_strategy":"primary","db_duration_s":0.002549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 235.878040] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.383Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c5b839cb08427b900c42f0a","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c5b839cb08427b900c42f0a: start","job_status":"start","queue_duration_s":0.710651,"scheduling_latency_s":0.710667}1441gitlab # [ 235.890774] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.393Z","message":"No existing merge request to be cleaned up."}1442gitlab # [ 235.892575] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.401Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c5b839cb08427b900c42f0a","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c5b839cb08427b900c42f0a: done: 0.018492 sec","job_status":"done","queue_duration_s":0.710651,"scheduling_latency_s":0.710667,"redis_calls":10,"redis_duration_s":0.00171,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018492,"completed_at":"2026-11-02T13:14:04.401Z","load_balancing_strategy":"primary","db_duration_s":0.001798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 235.933218] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.402Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4d80e0ce4ba0447af88841db","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4d80e0ce4ba0447af88841db: start","job_status":"start","queue_duration_s":0.730276,"scheduling_latency_s":0.730293}1444gitlab # [ 235.945820] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.413Z","message":"No existing merge request to be cleaned up."}1445gitlab # [ 235.947568] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.421Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4d80e0ce4ba0447af88841db","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4d80e0ce4ba0447af88841db: done: 0.018719 sec","job_status":"done","queue_duration_s":0.730276,"scheduling_latency_s":0.730293,"redis_calls":10,"redis_duration_s":0.001804,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018719,"completed_at":"2026-11-02T13:14:04.421Z","load_balancing_strategy":"primary","db_duration_s":0.001691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 235.987481] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.422Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5deb81eccd89af4833a9d352","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5deb81eccd89af4833a9d352: start","job_status":"start","queue_duration_s":0.750179,"scheduling_latency_s":0.750195}1447gitlab # [ 236.000208] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.433Z","message":"No existing merge request to be cleaned up."}1448gitlab # [ 236.001925] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.442Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5deb81eccd89af4833a9d352","created_at":"2026-11-02T13:14:03.662Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-85e059637f194e04","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"71a8e5bd80d0c60b57b4e1e85ab2fb85","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:03.672Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5deb81eccd89af4833a9d352: done: 0.019492 sec","job_status":"done","queue_duration_s":0.750179,"scheduling_latency_s":0.750195,"redis_calls":10,"redis_duration_s":0.0017529999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019492,"completed_at":"2026-11-02T13:14:04.442Z","load_balancing_strategy":"primary","db_duration_s":0.001745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 236.041436] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f66dda9794c602981665a01","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f66dda9794c602981665a01: start","job_status":"start","queue_duration_s":0.112149,"scheduling_latency_s":0.112164}1450gitlab # [ 236.054202] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f66dda9794c602981665a01","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f66dda9794c602981665a01: done: 0.015175 sec","job_status":"done","queue_duration_s":0.112149,"scheduling_latency_s":0.112164,"redis_calls":13,"redis_duration_s":0.001991,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001306,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015175,"completed_at":"2026-11-02T13:14:04.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 236.094504] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fcd5904a48be14111f249ec","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fcd5904a48be14111f249ec: start","job_status":"start","queue_duration_s":0.128476,"scheduling_latency_s":0.128492}1452gitlab # [ 236.107366] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fcd5904a48be14111f249ec","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fcd5904a48be14111f249ec: done: 0.015222 sec","job_status":"done","queue_duration_s":0.128476,"scheduling_latency_s":0.128492,"redis_calls":13,"redis_duration_s":0.002003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001329,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015222,"completed_at":"2026-11-02T13:14:04.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 236.149022] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a024b8f6b6f308f09178967","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a024b8f6b6f308f09178967: start","job_status":"start","queue_duration_s":0.144818,"scheduling_latency_s":0.144833}1454gitlab # [ 236.161736] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a024b8f6b6f308f09178967","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a024b8f6b6f308f09178967: done: 0.017014 sec","job_status":"done","queue_duration_s":0.144818,"scheduling_latency_s":0.144833,"redis_calls":13,"redis_duration_s":0.001975,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001329,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017014,"completed_at":"2026-11-02T13:14:04.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 236.201799] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7fdfc66e7c6c048dff6f06e","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7fdfc66e7c6c048dff6f06e: start","job_status":"start","queue_duration_s":0.162987,"scheduling_latency_s":0.163003}1456gitlab # [ 236.214599] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7fdfc66e7c6c048dff6f06e","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7fdfc66e7c6c048dff6f06e: done: 0.014934 sec","job_status":"done","queue_duration_s":0.162987,"scheduling_latency_s":0.163003,"redis_calls":13,"redis_duration_s":0.002001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001339,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014934,"completed_at":"2026-11-02T13:14:04.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 236.254707] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff3ed56e47596d2ec6aeeb02","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff3ed56e47596d2ec6aeeb02: start","job_status":"start","queue_duration_s":0.179066,"scheduling_latency_s":0.179082}1458gitlab # [ 236.267600] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff3ed56e47596d2ec6aeeb02","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff3ed56e47596d2ec6aeeb02: done: 0.021412 sec","job_status":"done","queue_duration_s":0.179066,"scheduling_latency_s":0.179082,"redis_calls":13,"redis_duration_s":0.0072569999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006548,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021412,"completed_at":"2026-11-02T13:14:04.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 236.308063] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45eb8e8daa6671d0e0d4b1ef","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45eb8e8daa6671d0e0d4b1ef: start","job_status":"start","queue_duration_s":0.20162,"scheduling_latency_s":0.201636}1460gitlab # [ 236.320504] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45eb8e8daa6671d0e0d4b1ef","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45eb8e8daa6671d0e0d4b1ef: done: 0.021663 sec","job_status":"done","queue_duration_s":0.20162,"scheduling_latency_s":0.201636,"redis_calls":13,"redis_duration_s":0.0064529999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005259,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021663,"completed_at":"2026-11-02T13:14:04.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 236.361782] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8e938887ff677017b6af809","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8e938887ff677017b6af809: start","job_status":"start","queue_duration_s":0.224413,"scheduling_latency_s":0.224429}1462gitlab # [ 236.374498] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8e938887ff677017b6af809","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8e938887ff677017b6af809: done: 0.015884 sec","job_status":"done","queue_duration_s":0.224413,"scheduling_latency_s":0.224429,"redis_calls":13,"redis_duration_s":0.002565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015884,"completed_at":"2026-11-02T13:14:04.571Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 236.414619] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75f319b71f379e7ddb2033e9","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75f319b71f379e7ddb2033e9: start","job_status":"start","queue_duration_s":0.241346,"scheduling_latency_s":0.241361}1464gitlab # [ 236.427564] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75f319b71f379e7ddb2033e9","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75f319b71f379e7ddb2033e9: done: 0.01449 sec","job_status":"done","queue_duration_s":0.241346,"scheduling_latency_s":0.241361,"redis_calls":13,"redis_duration_s":0.001675,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01449,"completed_at":"2026-11-02T13:14:04.586Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 236.467669] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f76596aad7ef310de1120d7b","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f76596aad7ef310de1120d7b: start","job_status":"start","queue_duration_s":0.256832,"scheduling_latency_s":0.256847}1466gitlab # [ 236.480331] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f76596aad7ef310de1120d7b","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f76596aad7ef310de1120d7b: done: 0.015104 sec","job_status":"done","queue_duration_s":0.256832,"scheduling_latency_s":0.256847,"redis_calls":13,"redis_duration_s":0.002199,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001329,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015104,"completed_at":"2026-11-02T13:14:04.603Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 236.520311] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80690916b1dad78b577cc60","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80690916b1dad78b577cc60: start","job_status":"start","queue_duration_s":0.273126,"scheduling_latency_s":0.273141}1468gitlab # [ 236.533026] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80690916b1dad78b577cc60","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80690916b1dad78b577cc60: done: 0.014938 sec","job_status":"done","queue_duration_s":0.273126,"scheduling_latency_s":0.273141,"redis_calls":13,"redis_duration_s":0.002086,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001385,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014938,"completed_at":"2026-11-02T13:14:04.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 236.574793] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e21a7853696d381bb6e5723","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e21a7853696d381bb6e5723: start","job_status":"start","queue_duration_s":0.289195,"scheduling_latency_s":0.28921}1470gitlab # [ 236.587615] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e21a7853696d381bb6e5723","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e21a7853696d381bb6e5723: done: 0.01509 sec","job_status":"done","queue_duration_s":0.289195,"scheduling_latency_s":0.28921,"redis_calls":13,"redis_duration_s":0.001852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000683,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01509,"completed_at":"2026-11-02T13:14:04.635Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 236.628124] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38c29d2d657e3f371adfde2f","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38c29d2d657e3f371adfde2f: start","job_status":"start","queue_duration_s":0.305393,"scheduling_latency_s":0.305409}1472gitlab # [ 236.640737] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38c29d2d657e3f371adfde2f","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38c29d2d657e3f371adfde2f: done: 0.014049 sec","job_status":"done","queue_duration_s":0.305393,"scheduling_latency_s":0.305409,"redis_calls":13,"redis_duration_s":0.001951,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014049,"completed_at":"2026-11-02T13:14:04.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 236.680932] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8dad3d2c9d850c9a93e8597","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8dad3d2c9d850c9a93e8597: start","job_status":"start","queue_duration_s":0.320881,"scheduling_latency_s":0.320897}1474gitlab # [ 236.693686] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8dad3d2c9d850c9a93e8597","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8dad3d2c9d850c9a93e8597: done: 0.014808 sec","job_status":"done","queue_duration_s":0.320881,"scheduling_latency_s":0.320897,"redis_calls":13,"redis_duration_s":0.002162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001481,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014808,"completed_at":"2026-11-02T13:14:04.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 236.734096] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb05688086065a4f6a29f724","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb05688086065a4f6a29f724: start","job_status":"start","queue_duration_s":0.33681,"scheduling_latency_s":0.336825}1476gitlab # [ 236.748009] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb05688086065a4f6a29f724","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb05688086065a4f6a29f724: done: 0.014225 sec","job_status":"done","queue_duration_s":0.33681,"scheduling_latency_s":0.336825,"redis_calls":13,"redis_duration_s":0.001831,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014225,"completed_at":"2026-11-02T13:14:04.682Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 236.787992] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6534f2b47a020d169068696","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6534f2b47a020d169068696: start","job_status":"start","queue_duration_s":0.352161,"scheduling_latency_s":0.352177}1478gitlab # [ 236.800643] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6534f2b47a020d169068696","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6534f2b47a020d169068696: done: 0.014144 sec","job_status":"done","queue_duration_s":0.352161,"scheduling_latency_s":0.352177,"redis_calls":13,"redis_duration_s":0.001879,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014144,"completed_at":"2026-11-02T13:14:04.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 236.840824] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cd7f2051a3bf127d95fbce5","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cd7f2051a3bf127d95fbce5: start","job_status":"start","queue_duration_s":0.367555,"scheduling_latency_s":0.367571}1480gitlab # [ 236.853616] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cd7f2051a3bf127d95fbce5","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cd7f2051a3bf127d95fbce5: done: 0.014245 sec","job_status":"done","queue_duration_s":0.367555,"scheduling_latency_s":0.367571,"redis_calls":13,"redis_duration_s":0.001875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014245,"completed_at":"2026-11-02T13:14:04.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 236.893535] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aa56d6d96f1f8f0b783c27e","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aa56d6d96f1f8f0b783c27e: start","job_status":"start","queue_duration_s":0.383084,"scheduling_latency_s":0.3831}1482gitlab # [ 236.906266] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9aa56d6d96f1f8f0b783c27e","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9aa56d6d96f1f8f0b783c27e: done: 0.014698 sec","job_status":"done","queue_duration_s":0.383084,"scheduling_latency_s":0.3831,"redis_calls":13,"redis_duration_s":0.001947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014698,"completed_at":"2026-11-02T13:14:04.728Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 236.946469] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c6b08fcfd895202109d15c0","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c6b08fcfd895202109d15c0: start","job_status":"start","queue_duration_s":0.398952,"scheduling_latency_s":0.398968}1484gitlab # [ 236.960724] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c6b08fcfd895202109d15c0","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c6b08fcfd895202109d15c0: done: 0.014275 sec","job_status":"done","queue_duration_s":0.398952,"scheduling_latency_s":0.398968,"redis_calls":13,"redis_duration_s":0.0017959999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014275,"completed_at":"2026-11-02T13:14:04.744Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 237.001036] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1835aee8d63e0c31fc487b21","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1835aee8d63e0c31fc487b21: start","job_status":"start","queue_duration_s":0.414345,"scheduling_latency_s":0.41436}1486gitlab # [ 237.013756] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1835aee8d63e0c31fc487b21","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1835aee8d63e0c31fc487b21: done: 0.015894 sec","job_status":"done","queue_duration_s":0.414345,"scheduling_latency_s":0.41436,"redis_calls":13,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015894,"completed_at":"2026-11-02T13:14:04.761Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 237.054080] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2385637bf73050682f6f434a","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2385637bf73050682f6f434a: start","job_status":"start","queue_duration_s":0.431622,"scheduling_latency_s":0.431638}1488gitlab # [ 237.066812] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2385637bf73050682f6f434a","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2385637bf73050682f6f434a: done: 0.014845 sec","job_status":"done","queue_duration_s":0.431622,"scheduling_latency_s":0.431638,"redis_calls":13,"redis_duration_s":0.0019450000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001274,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014845,"completed_at":"2026-11-02T13:14:04.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 237.107109] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41d6f821865f3191f5dc6d2d","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41d6f821865f3191f5dc6d2d: start","job_status":"start","queue_duration_s":0.447713,"scheduling_latency_s":0.447729}1490gitlab # [ 237.120073] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41d6f821865f3191f5dc6d2d","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41d6f821865f3191f5dc6d2d: done: 0.014787 sec","job_status":"done","queue_duration_s":0.447713,"scheduling_latency_s":0.447729,"redis_calls":13,"redis_duration_s":0.001974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014787,"completed_at":"2026-11-02T13:14:04.793Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 237.161339] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13932d95074dde3766e2f8f4","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13932d95074dde3766e2f8f4: start","job_status":"start","queue_duration_s":0.463652,"scheduling_latency_s":0.463668}1492gitlab # [ 237.174159] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13932d95074dde3766e2f8f4","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13932d95074dde3766e2f8f4: done: 0.024228 sec","job_status":"done","queue_duration_s":0.463652,"scheduling_latency_s":0.463668,"redis_calls":13,"redis_duration_s":0.010934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.008411,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002523,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024228,"completed_at":"2026-11-02T13:14:04.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 237.214116] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10b2527eee8b76fa9371dcc5","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10b2527eee8b76fa9371dcc5: start","job_status":"start","queue_duration_s":0.489759,"scheduling_latency_s":0.489777}1494gitlab # [ 237.226956] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10b2527eee8b76fa9371dcc5","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10b2527eee8b76fa9371dcc5: done: 0.015085 sec","job_status":"done","queue_duration_s":0.489759,"scheduling_latency_s":0.489777,"redis_calls":13,"redis_duration_s":0.002057,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015085,"completed_at":"2026-11-02T13:14:04.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 237.267180] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa785fe0449cfcff71378a92","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa785fe0449cfcff71378a92: start","job_status":"start","queue_duration_s":0.505973,"scheduling_latency_s":0.505989}1496gitlab # [ 237.280002] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa785fe0449cfcff71378a92","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa785fe0449cfcff71378a92: done: 0.014311 sec","job_status":"done","queue_duration_s":0.505973,"scheduling_latency_s":0.505989,"redis_calls":13,"redis_duration_s":0.0017749999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014311,"completed_at":"2026-11-02T13:14:04.851Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 237.320155] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3caa1212c860b60c31e3374","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3caa1212c860b60c31e3374: start","job_status":"start","queue_duration_s":0.521414,"scheduling_latency_s":0.521429}1498gitlab # [ 237.332872] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3caa1212c860b60c31e3374","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3caa1212c860b60c31e3374: done: 0.014781 sec","job_status":"done","queue_duration_s":0.521414,"scheduling_latency_s":0.521429,"redis_calls":13,"redis_duration_s":0.00212,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014781,"completed_at":"2026-11-02T13:14:04.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 237.374180] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de6af4d0426720ff99f17807","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de6af4d0426720ff99f17807: start","job_status":"start","queue_duration_s":0.53812,"scheduling_latency_s":0.538136}1500gitlab # [ 237.386742] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de6af4d0426720ff99f17807","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de6af4d0426720ff99f17807: done: 0.01519 sec","job_status":"done","queue_duration_s":0.53812,"scheduling_latency_s":0.538136,"redis_calls":13,"redis_duration_s":0.001993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001301,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01519,"completed_at":"2026-11-02T13:14:04.884Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 237.427035] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"813078a141d51199445343fd","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-813078a141d51199445343fd: start","job_status":"start","queue_duration_s":0.554543,"scheduling_latency_s":0.554559}1502gitlab # [ 237.439962] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"813078a141d51199445343fd","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-813078a141d51199445343fd: done: 0.014311 sec","job_status":"done","queue_duration_s":0.554543,"scheduling_latency_s":0.554559,"redis_calls":13,"redis_duration_s":0.0018670000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014311,"completed_at":"2026-11-02T13:14:04.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 237.480351] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf8f6e05fd94827f02b5aad","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf8f6e05fd94827f02b5aad: start","job_status":"start","queue_duration_s":0.570027,"scheduling_latency_s":0.570042}1504gitlab # [ 237.493089] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf8f6e05fd94827f02b5aad","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf8f6e05fd94827f02b5aad: done: 0.014179 sec","job_status":"done","queue_duration_s":0.570027,"scheduling_latency_s":0.570042,"redis_calls":13,"redis_duration_s":0.001876,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001226,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014179,"completed_at":"2026-11-02T13:14:04.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 237.533038] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fde89dab1056356ade7e8fc","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fde89dab1056356ade7e8fc: start","job_status":"start","queue_duration_s":0.585544,"scheduling_latency_s":0.585559}1506gitlab # [ 237.545769] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fde89dab1056356ade7e8fc","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fde89dab1056356ade7e8fc: done: 0.014422 sec","job_status":"done","queue_duration_s":0.585544,"scheduling_latency_s":0.585559,"redis_calls":13,"redis_duration_s":0.001839,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014422,"completed_at":"2026-11-02T13:14:04.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 237.587561] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5267ca0169519a9e6110c075","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5267ca0169519a9e6110c075: start","job_status":"start","queue_duration_s":0.601152,"scheduling_latency_s":0.601167}1508gitlab # [ 237.600612] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5267ca0169519a9e6110c075","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5267ca0169519a9e6110c075: done: 0.014891 sec","job_status":"done","queue_duration_s":0.601152,"scheduling_latency_s":0.601167,"redis_calls":13,"redis_duration_s":0.002069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014891,"completed_at":"2026-11-02T13:14:04.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 237.640801] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c93a1e5cb1f07bca342a7888","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c93a1e5cb1f07bca342a7888: start","job_status":"start","queue_duration_s":0.617264,"scheduling_latency_s":0.61728}1510gitlab # [ 237.653556] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c93a1e5cb1f07bca342a7888","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c93a1e5cb1f07bca342a7888: done: 0.014403 sec","job_status":"done","queue_duration_s":0.617264,"scheduling_latency_s":0.61728,"redis_calls":13,"redis_duration_s":0.001884,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014403,"completed_at":"2026-11-02T13:14:04.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 237.693664] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d49547a7e3d411a4c45fc49","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d49547a7e3d411a4c45fc49: start","job_status":"start","queue_duration_s":0.633264,"scheduling_latency_s":0.63328}1512gitlab # [ 237.706490] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d49547a7e3d411a4c45fc49","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d49547a7e3d411a4c45fc49: done: 0.014664 sec","job_status":"done","queue_duration_s":0.633264,"scheduling_latency_s":0.63328,"redis_calls":13,"redis_duration_s":0.001912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014664,"completed_at":"2026-11-02T13:14:04.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 237.746857] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e14de3acca9639bad763f4c","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e14de3acca9639bad763f4c: start","job_status":"start","queue_duration_s":0.649392,"scheduling_latency_s":0.649408}1514gitlab # [ 237.759836] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e14de3acca9639bad763f4c","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e14de3acca9639bad763f4c: done: 0.014642 sec","job_status":"done","queue_duration_s":0.649392,"scheduling_latency_s":0.649408,"redis_calls":13,"redis_duration_s":0.0018219999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014642,"completed_at":"2026-11-02T13:14:04.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 237.801649] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:04.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a62c0e54d518ee6287add559","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a62c0e54d518ee6287add559: start","job_status":"start","queue_duration_s":0.66522,"scheduling_latency_s":0.665236}1516gitlab # [ 237.814365] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a62c0e54d518ee6287add559","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a62c0e54d518ee6287add559: done: 0.014763 sec","job_status":"done","queue_duration_s":0.66522,"scheduling_latency_s":0.665236,"redis_calls":13,"redis_duration_s":0.002186,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001476,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014763,"completed_at":"2026-11-02T13:14:05.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 237.854627] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0adf313edc81fb994ed03f5","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0adf313edc81fb994ed03f5: start","job_status":"start","queue_duration_s":0.681181,"scheduling_latency_s":0.681196}1518gitlab # [ 237.867544] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a0adf313edc81fb994ed03f5","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a0adf313edc81fb994ed03f5: done: 0.01483 sec","job_status":"done","queue_duration_s":0.681181,"scheduling_latency_s":0.681196,"redis_calls":13,"redis_duration_s":0.001989,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01483,"completed_at":"2026-11-02T13:14:05.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 237.907949] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76ed01c5f38af240f030d27c","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76ed01c5f38af240f030d27c: start","job_status":"start","queue_duration_s":0.697171,"scheduling_latency_s":0.697187}1520gitlab # [ 237.920909] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76ed01c5f38af240f030d27c","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76ed01c5f38af240f030d27c: done: 0.01437 sec","job_status":"done","queue_duration_s":0.697171,"scheduling_latency_s":0.697187,"redis_calls":13,"redis_duration_s":0.001786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01437,"completed_at":"2026-11-02T13:14:05.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 237.960976] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6d77a17daf8efa5b0d4bfcb","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6d77a17daf8efa5b0d4bfcb: start","job_status":"start","queue_duration_s":0.71289,"scheduling_latency_s":0.712906}1522gitlab # [ 237.974767] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6d77a17daf8efa5b0d4bfcb","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6d77a17daf8efa5b0d4bfcb: done: 0.014388 sec","job_status":"done","queue_duration_s":0.71289,"scheduling_latency_s":0.712906,"redis_calls":13,"redis_duration_s":0.001824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014388,"completed_at":"2026-11-02T13:14:05.058Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 238.014934] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72c8952b131d0b332cac3082","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72c8952b131d0b332cac3082: start","job_status":"start","queue_duration_s":0.728449,"scheduling_latency_s":0.728464}1524gitlab # [ 238.027896] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72c8952b131d0b332cac3082","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72c8952b131d0b332cac3082: done: 0.014776 sec","job_status":"done","queue_duration_s":0.728449,"scheduling_latency_s":0.728464,"redis_calls":13,"redis_duration_s":0.002148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001479,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014776,"completed_at":"2026-11-02T13:14:05.074Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 238.068527] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e98531e2a9c7cccedcc20ff","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e98531e2a9c7cccedcc20ff: start","job_status":"start","queue_duration_s":0.744391,"scheduling_latency_s":0.744406}1526gitlab # [ 238.081062] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e98531e2a9c7cccedcc20ff","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e98531e2a9c7cccedcc20ff: done: 0.014365 sec","job_status":"done","queue_duration_s":0.744391,"scheduling_latency_s":0.744406,"redis_calls":13,"redis_duration_s":0.0018870000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014365,"completed_at":"2026-11-02T13:14:05.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 238.121588] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83a8d44e33ece2e77b1fe184","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83a8d44e33ece2e77b1fe184: start","job_status":"start","queue_duration_s":0.760188,"scheduling_latency_s":0.760204}1528gitlab # [ 238.134333] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83a8d44e33ece2e77b1fe184","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83a8d44e33ece2e77b1fe184: done: 0.014845 sec","job_status":"done","queue_duration_s":0.760188,"scheduling_latency_s":0.760204,"redis_calls":13,"redis_duration_s":0.002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001304,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014845,"completed_at":"2026-11-02T13:14:05.106Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 238.174925] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afaedff433d7e82ddc49e1e7","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afaedff433d7e82ddc49e1e7: start","job_status":"start","queue_duration_s":0.776194,"scheduling_latency_s":0.77621}1530gitlab # [ 238.187820] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afaedff433d7e82ddc49e1e7","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afaedff433d7e82ddc49e1e7: done: 0.014338 sec","job_status":"done","queue_duration_s":0.776194,"scheduling_latency_s":0.77621,"redis_calls":13,"redis_duration_s":0.001889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014338,"completed_at":"2026-11-02T13:14:05.121Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 238.227980] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"432ee3858b870e9046ae3082","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-432ee3858b870e9046ae3082: start","job_status":"start","queue_duration_s":0.791692,"scheduling_latency_s":0.791708}1532gitlab # [ 238.240873] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"432ee3858b870e9046ae3082","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-432ee3858b870e9046ae3082: done: 0.014247 sec","job_status":"done","queue_duration_s":0.791692,"scheduling_latency_s":0.791708,"redis_calls":13,"redis_duration_s":0.001852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014247,"completed_at":"2026-11-02T13:14:05.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 238.280718] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e68d98d7f1263f4b3756356","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e68d98d7f1263f4b3756356: start","job_status":"start","queue_duration_s":0.807514,"scheduling_latency_s":0.807529}1534gitlab # [ 238.293419] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e68d98d7f1263f4b3756356","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e68d98d7f1263f4b3756356: done: 0.014638 sec","job_status":"done","queue_duration_s":0.807514,"scheduling_latency_s":0.807529,"redis_calls":13,"redis_duration_s":0.001836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014638,"completed_at":"2026-11-02T13:14:05.153Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 238.333160] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9388e9749c47518dbabad301","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9388e9749c47518dbabad301: start","job_status":"start","queue_duration_s":0.823302,"scheduling_latency_s":0.823318}1536gitlab # [ 238.345613] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9388e9749c47518dbabad301","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9388e9749c47518dbabad301: done: 0.014397 sec","job_status":"done","queue_duration_s":0.823302,"scheduling_latency_s":0.823318,"redis_calls":13,"redis_duration_s":0.002003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014397,"completed_at":"2026-11-02T13:14:05.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 238.386621] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc6e3ad06f34384bbdd1f6ed","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc6e3ad06f34384bbdd1f6ed: start","job_status":"start","queue_duration_s":0.838867,"scheduling_latency_s":0.838899}1538gitlab # [ 238.399526] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc6e3ad06f34384bbdd1f6ed","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc6e3ad06f34384bbdd1f6ed: done: 0.014337 sec","job_status":"done","queue_duration_s":0.838867,"scheduling_latency_s":0.838899,"redis_calls":13,"redis_duration_s":0.0018960000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014337,"completed_at":"2026-11-02T13:14:05.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 238.439743] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"965f61d40556a50992811019","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-965f61d40556a50992811019: start","job_status":"start","queue_duration_s":0.854473,"scheduling_latency_s":0.854489}1540gitlab # [ 238.452509] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"965f61d40556a50992811019","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-965f61d40556a50992811019: done: 0.014748 sec","job_status":"done","queue_duration_s":0.854473,"scheduling_latency_s":0.854489,"redis_calls":13,"redis_duration_s":0.001895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014748,"completed_at":"2026-11-02T13:14:05.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 238.492555] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73b5fb936ac3adf941d56734","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73b5fb936ac3adf941d56734: start","job_status":"start","queue_duration_s":0.870406,"scheduling_latency_s":0.870422}1542gitlab # [ 238.505180] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73b5fb936ac3adf941d56734","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73b5fb936ac3adf941d56734: done: 0.014207 sec","job_status":"done","queue_duration_s":0.870406,"scheduling_latency_s":0.870422,"redis_calls":13,"redis_duration_s":0.002041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001405,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014207,"completed_at":"2026-11-02T13:14:05.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 238.545163] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e176e205ae96b6e6ed9c8b4","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e176e205ae96b6e6ed9c8b4: start","job_status":"start","queue_duration_s":0.885859,"scheduling_latency_s":0.885875}1544gitlab # [ 238.558012] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e176e205ae96b6e6ed9c8b4","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e176e205ae96b6e6ed9c8b4: done: 0.014357 sec","job_status":"done","queue_duration_s":0.885859,"scheduling_latency_s":0.885875,"redis_calls":13,"redis_duration_s":0.0018869999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014357,"completed_at":"2026-11-02T13:14:05.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 238.600589] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c220063a9dcb9f981c916ad4","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c220063a9dcb9f981c916ad4: start","job_status":"start","queue_duration_s":0.902416,"scheduling_latency_s":0.902433}1546gitlab # [ 238.620455] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c220063a9dcb9f981c916ad4","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c220063a9dcb9f981c916ad4: done: 0.016658 sec","job_status":"done","queue_duration_s":0.902416,"scheduling_latency_s":0.902433,"redis_calls":13,"redis_duration_s":0.002026,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016658,"completed_at":"2026-11-02T13:14:05.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 238.681012] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf4873fa08cf2234996a86b0","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf4873fa08cf2234996a86b0: start","job_status":"start","queue_duration_s":0.920291,"scheduling_latency_s":0.920306}1548gitlab # [ 238.696478] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:14:05.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf4873fa08cf2234996a86b0","created_at":"2026-11-02T13:14:04.255Z","trace_propagation_headers":{"sentry-trace":"c83fda453669421e81966ec45ee94a2e-40a56cb576fb4fb2","baggage":"sentry-trace_id=c83fda453669421e81966ec45ee94a2e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"87d3f08f872408e1f132d52e52bee7e8","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:14:04.331Z","job_size_bytes":2,"pid":1873,"sidekiq_tid":"mam1","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf4873fa08cf2234996a86b0: done: 0.016525 sec","job_status":"done","queue_duration_s":0.920291,"scheduling_latency_s":0.920306,"redis_calls":13,"redis_duration_s":0.0023,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001663,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016525,"completed_at":"2026-11-02T13:14:05.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 247.016557] setup-gitlab-start[2818]: W, [2026-11-02T13:14:15.997769 #2818] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1550gitlab # [ 286.121292] postgres[3158]: [3158] ERROR: relation "table_without_models" does not exist1551gitlab # [ 286.121668] postgres[3158]: [3158] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1552gitlab # [ 291.741032] runuser[2817]: pam_unix(runuser:session): session closed for user gitlab1553gitlab # [ 291.741992] setup-gitlab-start[2791]: + systemctl restart gitlab-sidekiq.service1554gitlab # [ 291.767049] systemd[1]: Stopping gitlab-sidekiq.service...1555gitlab # [ 291.768965] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:15:00.748Z","message":"Shutting down"}1556gitlab # [ 291.771237] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:15:00.749Z","message":"Scheduler exiting..."}1557gitlab # [ 291.773572] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:15:00.749Z","message":"Terminating quiet threads for default capsule"}1558gitlab # [ 291.776280] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:15:00.750Z","message":"Scheduler exiting..."}1559gitlab # [ 292.275886] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:15:01.257Z","message":"Pausing to allow jobs to finish..."}1560gitlab # [ 296.779635] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:15:05.760Z","message":"Bye!"}1561gitlab # [ 296.806906] sidekiq-cluster[1873]: {"severity":"INFO","time":"2026-11-02T13:15:05.788Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1873,"worker_id":"sidekiq_0","memwd_rss_bytes":920100864,"retry":0}1562gitlab # [ 297.295049] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1563gitlab # [ 297.296746] systemd[1]: Stopped gitlab-sidekiq.service.1564gitlab # [ 297.297966] systemd[1]: gitlab-sidekiq.service: Consumed 1min 44.848s CPU time over 3min 17.333s wall clock time, 939.2M memory peak, 5M read from disk.1565gitlab # [ 297.301157] systemd[1]: Started gitlab-sidekiq.service.1566gitlab # [ 297.302818] setup-gitlab-start[2791]: + mkdir -p /var/lib/secrets1567gitlab # [ 297.304353] setup-gitlab-start[2791]: + echo nixbot-test-token-of-20-chars1568gitlab # [ 297.305839] setup-gitlab-start[2791]: + chmod 644 /var/lib/secrets/gitlab-token1569gitlab # [ 297.307431] setup-gitlab-start[2791]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1570gitlab # [ 298.052909] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:07 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 7211571gitlab # [ 298.756576] gitaly[920]: time="2026-11-02T13:15:07.737Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:17.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.737" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1572gitlab # [ 298.772757] gitaly[920]: time="2026-11-02T13:15:07.738Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:17.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.737" grpc.time_ms=0.396 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1573gitlab # [ 298.931964] gitaly[920]: time="2026-11-02T13:15:07.913Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:17.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.913" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1574gitlab # [ 298.950741] gitaly[920]: time="2026-11-02T13:15:07.914Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:17.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.913" grpc.time_ms=0.371 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1575gitlab # [ 298.985873] gitaly[920]: time="2026-11-02T13:15:07.967Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R diskcache=8d4ce37f-6b23-424a-a7ef-520f0b1f5311 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-02T13:15:17.200" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.930" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1576gitlab # [ 299.003525] gitaly[920]: time="2026-11-02T13:15:07.968Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-02T13:15:17.200" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.930" grpc.time_ms=36.989 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1577gitlab # [ 299.024977] gitaly[920]: time="2026-11-02T13:15:07.981Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:17.250" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.980" pid=920 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1578gitlab # [ 299.040295] gitaly[920]: time="2026-11-02T13:15:07.981Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:17.250" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.980" grpc.time_ms=0.768 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1579gitlab # [ 299.054667] gitaly[920]: time="2026-11-02T13:15:07.997Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:15:17.256" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:07.986" grpc.time_ms=10.801 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1580gitlab # [ 299.088256] gitaly[920]: time="2026-11-02T13:15:08.070Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:18.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.069" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1581gitlab # [ 299.102824] gitaly[920]: time="2026-11-02T13:15:08.070Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:18.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.069" grpc.time_ms=0.376 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1582gitlab # [ 299.119237] gitaly[920]: time="2026-11-02T13:15:08.100Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R diskcache=07fd956a-ad62-4b7a-b106-9c936792c79b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-02T13:15:18.011" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.072" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 299.133489] gitaly[920]: time="2026-11-02T13:15:08.101Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-02T13:15:18.011" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.072" grpc.time_ms=28.859 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 299.149909] gitaly[920]: time="2026-11-02T13:15:08.106Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:18.045" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.105" pid=920 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1585gitlab # [ 299.161668] gitaly[920]: time="2026-11-02T13:15:08.106Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:18.045" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.105" grpc.time_ms=0.82 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586gitlab # [ 299.173565] gitaly[920]: time="2026-11-02T13:15:08.118Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:15:18.048" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.109" grpc.time_ms=9.122 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1587gitlab # [ 299.188159] gitaly[920]: time="2026-11-02T13:15:08.144Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:15:18.083" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.143" grpc.time_ms=0.13 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1588gitlab # [ 299.200209] gitaly[920]: time="2026-11-02T13:15:08.150Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-02T13:15:18.089" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.150" grpc.time_ms=0.244 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1589gitlab # [ 299.685616] gitaly[920]: time="2026-11-02T13:15:08.667Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CDXEPZ27XA2SEY0RY79R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:15:18.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:08.657" grpc.time_ms=9.977 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 300.081548] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:09 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 20071591gitlab # [ 300.084544] setup-gitlab-start[3233]: {"id":1,"message":null,"error":null}1592gitlab # [ 302.430382] sidekiq-cluster[3221]: {"severity":"INFO","time":"2026-11-02T13:15:11.374Z","message":"Starting cluster with 1 processes"}1593gitlab # [ 305.227927] gitaly[920]: time="2026-11-02T13:15:14.209Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CMS6WF06AX2E4229YAEQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:15:24.138" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:14.199" grpc.time_ms=9.588 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594gitlab # [ 305.303723] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:14 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1981595gitlab # [ 305.307584] setup-gitlab-start[2791]: + mkdir -p /tmp/test-flake1596gitlab # [ 305.310152] setup-gitlab-start[2791]: + cd /tmp/test-flake1597gitlab # [ 305.311307] setup-gitlab-start[2791]: + git init -b master1598gitlab # [ 305.331490] setup-gitlab-start[3306]: Initialized empty Git repository in /tmp/test-flake/.git/1599gitlab # [ 305.333464] setup-gitlab-start[2791]: + git config user.name test1600gitlab # [ 305.336100] setup-gitlab-start[2791]: + git config user.email test@example.com1601gitlab # [ 305.340038] setup-gitlab-start[2791]: + cat1602gitlab # [ 305.343156] setup-gitlab-start[2791]: + git add flake.nix1603gitlab # [ 305.347738] setup-gitlab-start[2791]: + git commit -m 'initial commit'1604gitlab # [ 305.369566] setup-gitlab-start[3311]: [master (root-commit) 1012c45] initial commit1605gitlab # [ 305.371301] setup-gitlab-start[3311]: 1 file changed, 10 insertions(+)1606gitlab # [ 305.372777] setup-gitlab-start[3311]: create mode 100644 flake.nix1607gitlab # [ 305.375099] setup-gitlab-start[2791]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1608gitlab # [ 305.382155] setup-gitlab-start[2791]: + git push -u origin master1609gitlab # [ 305.582128] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1711610gitlab # [ 305.843117] gitaly[920]: time="2026-11-02T13:15:14.824Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CN84XHSGQTF9H0JTD207 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:14.813" grpc.time_ms=11.113 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1611gitlab # [ 305.865183] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 2601612gitlab # [ 306.141699] gitaly[920]: time="2026-11-02T13:15:15.123Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:15:25.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:15.114" grpc.time_ms=8.713 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1613gitlab # [ 306.183903] gitaly[920]: time="2026-11-02T13:15:15.165Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-02T13:15:45.055" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-02T13:15:15.155" grpc.time_ms=10.068 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 306.246533] gitaly[920]: time="2026-11-02T13:15:15.227Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N duration_ms=261ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/allowed"1615gitlab # [ 306.280177] gitaly[920]: time="2026-11-02T13:15:15.262Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N duration_ms=33ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/pre_receive"1616gitlab # [ 306.284838] gitaly[920]: time="2026-11-02T13:15:15.262Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:15:14.965" grpc.time_ms=297.233 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1617gitlab # [ 306.312584] gitaly[920]: time="2026-11-02T13:15:15.294Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:15:15.293" grpc.time_ms=0.706 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1618gitlab # [ 306.359388] gitaly[920]: time="2026-11-02T13:15:15.341Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:15:15.339" grpc.time_ms=2.011 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1619gitlab # [ 306.395940] gitaly[920]: time="2026-11-02T13:15:15.375Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:15:15.375" grpc.time_ms=0.507 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1620gitlab # [ 306.532865] gitaly[920]: time="2026-11-02T13:15:15.514Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:15:25.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:15.504" grpc.time_ms=10.422 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1621gitlab # [ 306.561812] gitaly[920]: time="2026-11-02T13:15:15.543Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1434 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-02T13:15:25.021" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:15.521" grpc.time_ms=21.729 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1622gitlab # [ 306.584250] gitaly[920]: time="2026-11-02T13:15:15.553Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N duration_ms=147ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/post_receive"1623gitlab # [ 306.588882] gitaly[920]: time="2026-11-02T13:15:15.555Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:15:15.405" grpc.time_ms=149.088 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1624gitlab # [ 306.605418] gitaly[920]: time="2026-11-02T13:15:15.562Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N diskcache=59dd02e6-49bd-4b53-8c40-7568d989ab16 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:14.904" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1625gitlab # [ 306.619898] setup-gitlab-start[3319]: To http://gitlab/root/test-flake.git1626gitlab # [ 306.621187] setup-gitlab-start[3319]: * [new branch] master -> master1627gitlab # [ 306.622366] setup-gitlab-start[3319]: branch 'master' set up to track 'origin/master'.1628gitlab # [ 306.624178] gitlab-workhorse[826]: time="2026-11-02T13:15:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521629gitlab # [ 306.626499] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:15 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7271630gitlab # [ 306.629105] systemd[1]: Finished setup-gitlab.service.1631gitlab # [ 306.630231] gitaly[920]: time="2026-11-02T13:15:15.562Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=184 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=12957 command.oublock=32 command.real_time_ms=651 command.spawn_token_fork_ms=0 command.system_time_ms=116 command.user_time_ms=68 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:14.904" grpc.time_ms=657.567 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1632gitlab # [ 306.645559] systemd[1]: Starting nixbot CI service...1633gitlab: (finished: waiting for unit setup-gitlab.service, in 308.03 seconds)1634gitlab: waiting for unit nixbot.service1635gitlab # [ 307.766339] nixbot[3407]: {"ts": "2026-11-02T13:15:16.747Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1636gitlab # [ 307.792168] nixbot[3407]: {"ts": "2026-11-02T13:15:16.773Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1637gitlab # [ 307.814351] nixbot[3407]: {"ts": "2026-11-02T13:15:16.796Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1638gitlab # [ 307.818831] nixbot[3407]: {"ts": "2026-11-02T13:15:16.800Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1639gitlab # [ 307.823028] nixbot[3407]: {"ts": "2026-11-02T13:15:16.804Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1640gitlab # [ 307.829173] nixbot[3407]: {"ts": "2026-11-02T13:15:16.811Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1641gitlab # [ 307.832686] nixbot[3407]: {"ts": "2026-11-02T13:15:16.814Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1642gitlab # [ 307.836146] nixbot[3407]: {"ts": "2026-11-02T13:15:16.815Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1643gitlab # [ 307.840532] nixbot[3407]: {"ts": "2026-11-02T13:15:16.822Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1644gitlab # [ 307.843980] nixbot[3407]: {"ts": "2026-11-02T13:15:16.824Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1645gitlab # [ 307.847647] nixbot[3407]: {"ts": "2026-11-02T13:15:16.829Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1646gitlab # [ 307.854272] nixbot[3407]: {"ts": "2026-11-02T13:15:16.836Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1647gitlab # [ 307.857746] nixbot[3407]: {"ts": "2026-11-02T13:15:16.839Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1648gitlab # [ 307.861494] nixbot[3407]: {"ts": "2026-11-02T13:15:16.840Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1649gitlab # [ 307.865182] nixbot[3407]: {"ts": "2026-11-02T13:15:16.844Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1650gitlab # [ 307.868775] nixbot[3407]: {"ts": "2026-11-02T13:15:16.845Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1651gitlab # [ 307.872245] nixbot[3407]: {"ts": "2026-11-02T13:15:16.847Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1652gitlab # [ 307.875771] nixbot[3407]: {"ts": "2026-11-02T13:15:16.848Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1653gitlab # [ 307.879263] nixbot[3407]: {"ts": "2026-11-02T13:15:16.852Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1654gitlab # [ 307.882939] nixbot[3407]: {"ts": "2026-11-02T13:15:16.853Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1655gitlab # [ 307.886605] nixbot[3407]: {"ts": "2026-11-02T13:15:16.854Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1656gitlab # [ 307.890025] nixbot[3407]: {"ts": "2026-11-02T13:15:16.859Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1657gitlab # [ 307.893374] nixbot[3407]: {"ts": "2026-11-02T13:15:16.860Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1658gitlab # [ 307.896559] nixbot[3407]: {"ts": "2026-11-02T13:15:16.862Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1659gitlab # [ 307.899569] nixbot[3407]: {"ts": "2026-11-02T13:15:16.863Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1660gitlab # [ 307.902812] nixbot[3407]: {"ts": "2026-11-02T13:15:16.865Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1661gitlab # [ 307.906115] nixbot[3407]: {"ts": "2026-11-02T13:15:16.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1662gitlab # [ 307.909298] nixbot[3407]: {"ts": "2026-11-02T13:15:16.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1663gitlab # [ 307.912196] nixbot[3407]: {"ts": "2026-11-02T13:15:16.868Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1664gitlab # [ 307.915193] nixbot[3407]: {"ts": "2026-11-02T13:15:16.880Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1665gitlab # [ 307.918175] nixbot[3407]: {"ts": "2026-11-02T13:15:16.884Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1666gitlab # [ 307.920977] nixbot[3407]: {"ts": "2026-11-02T13:15:16.886Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1667gitlab # [ 307.923804] nixbot[3407]: {"ts": "2026-11-02T13:15:16.887Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1668gitlab # [ 307.926678] nixbot[3407]: {"ts": "2026-11-02T13:15:16.891Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1669gitlab # [ 307.929604] nixbot[3407]: {"ts": "2026-11-02T13:15:16.892Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1670gitlab # [ 307.932567] nixbot[3407]: {"ts": "2026-11-02T13:15:16.897Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1671gitlab # [ 307.935253] nixbot[3407]: {"ts": "2026-11-02T13:15:16.899Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1672gitlab # [ 307.937873] nixbot[3407]: {"ts": "2026-11-02T13:15:16.902Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1673gitlab # [ 307.940497] nixbot[3407]: {"ts": "2026-11-02T13:15:16.904Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"}1674gitlab # [ 308.036335] redis[800]: 10 changes in 300 seconds. Saving...1675gitlab # [ 308.037972] redis[800]: Background saving started by pid 34341676gitlab # [ 308.042431] redis[3434]: BGSAVE done, 409 keys saved, 0 keys skipped, 107421 bytes written.1677gitlab # [ 308.045407] redis[3434]: DB saved on disk1678gitlab # [ 308.046633] redis[3434]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1679gitlab # [ 308.137091] redis[800]: Background saving terminated with success1680gitlab # [ 308.317461] nixbot[3407]: {"ts": "2026-11-02T13:15:17.299Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [3407]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1681gitlab # [ 308.321813] nixbot[3407]: {"ts": "2026-11-02T13:15:17.299Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1682gitlab # [ 308.354702] nixbot[3407]: {"ts": "2026-11-02T13:15:17.336Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:45282 - \"GET /health HTTP/1.1\" 200"}1683gitlab # [ 308.360957] systemd[1]: Started nixbot CI service.1684gitlab # [ 308.362093] systemd[1]: Reached target Multi-User System.1685gitlab # [ 308.363327] systemd[1]: Startup finished in 597ms (kernel) + 3.557s (initrd) + 5min 4.207s (userspace) = 5min 8.362s.1686gitlab: (finished: waiting for unit nixbot.service, in 1.06 seconds)1687gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1688gitlab # [ 308.671195] nixbot[3407]: {"ts": "2026-11-02T13:15:17.652Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:45292 - \"GET /health HTTP/1.1\" 200"}1689gitlab # [ 308.676774] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:17 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 3171690gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1691subtest: project discovered and webhook registered1692gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1693gitlab # [ 308.702945] nixbot[3407]: {"ts": "2026-11-02T13:15:17.684Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1694gitlab # [ 308.779747] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 651695gitlab # [ 308.785577] nixbot[3407]: {"ts": "2026-11-02T13:15:17.763Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1696gitlab # [ 308.828550] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 1251697gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.15 seconds)1698gitlab # [ 309.140117] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 3551699gitlab # [ 309.311317] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1621700gitlab # [ 309.443618] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 1221701gitlab # [ 309.539837] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 901702gitlab # [ 309.631204] gitaly[920]: time="2026-11-02T13:15:18.612Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CS41HSPG2FM894X9NJAR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-02T13:15:48.496" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:18.596" grpc.time_ms=15.136 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1703gitlab # [ 309.656124] gitaly[920]: time="2026-11-02T13:15:18.632Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CS41HSPG2FM894X9NJAR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:15:48.531" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:15:18.631" grpc.time_ms=1.128 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704gitlab # [ 309.676137] gitaly[920]: time="2026-11-02T13:15:18.647Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CS41HSPG2FM894X9NJAR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-02T13:15:28.136" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:15:18.636" grpc.time_ms=10.358 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705gitlab # [ 309.695855] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1471706gitlab # [ 309.793982] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 931707gitlab # [ 309.800200] nixbot[3407]: {"ts": "2026-11-02T13:15:18.781Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1708gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1709gitlab # [ 309.892391] postgres[868]: [868] LOG: checkpoint starting: time1710gitlab # [ 309.940686] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 761711gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1712(finished: subtest: project discovered and webhook registered, in 1.27 seconds)1713subtest: push delivers the webhook1714gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1715gitlab # [ 309.969629] gitaly[920]: time="2026-11-02T13:15:18.951Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CSE98YR0KFRNTKCA09R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:18.939" grpc.time_ms=11.374 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1716gitlab # [ 309.984540] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 941717gitlab # [ 310.177109] gitaly[920]: time="2026-11-02T13:15:19.158Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSHBQVS95Y0KQAFD2BCC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.147" pid=920 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1718gitlab # [ 310.193746] gitaly[920]: time="2026-11-02T13:15:19.159Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSHBQVS95Y0KQAFD2BCC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.147" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1719gitlab # [ 310.215067] gitlab-workhorse[826]: time="2026-11-02T13:15:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131720gitlab # [ 310.218214] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 2041721gitlab # [ 310.221292] gitaly[920]: time="2026-11-02T13:15:19.159Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSHBQVS95Y0KQAFD2BCC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.147" grpc.time_ms=12.223 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1722gitlab # [ 310.264541] gitaly[920]: time="2026-11-02T13:15:19.246Z" level=info msg="burden monitor stats" pid=9201723gitlab # [ 310.270609] gitaly[920]: time="2026-11-02T13:15:19.252Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSQVK8D3JN3JTTCEZ30Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.235" pid=920 remote_ip="2001:db8:1::1" request_sha=dad538a0699c2ad0343fa2d0c118473a7b0cff14 response_bytes=224 span.kind=server system=grpc1724gitlab # [ 310.287739] gitaly[920]: time="2026-11-02T13:15:19.252Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1134 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSQVK8D3JN3JTTCEZ30Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.235" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1725gitlab # [ 310.308292] gitlab-workhorse[826]: time="2026-11-02T13:15:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2241726gitlab # [ 310.310949] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 224 "" "git/2.55.0" 1051727gitlab # [ 310.313722] gitaly[920]: time="2026-11-02T13:15:19.252Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1134 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSQVK8D3JN3JTTCEZ30Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.235" grpc.time_ms=16.851 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1728gitlab # [ 310.379655] gitaly[920]: time="2026-11-02T13:15:19.361Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CSW9DX8QHP94NYDR9CHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.351" grpc.time_ms=9.463 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1729gitlab # [ 310.401007] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 561730gitlab # [ 310.537925] gitaly[920]: time="2026-11-02T13:15:19.519Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSY3QSNBGVTZQ21SDGN7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.506" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1731gitlab # [ 310.554474] gitaly[920]: time="2026-11-02T13:15:19.519Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSY3QSNBGVTZQ21SDGN7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.506" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 310.576002] gitlab-workhorse[826]: time="2026-11-02T13:15:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671733gitlab # [ 310.578939] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1571734gitlab # [ 310.581826] gitaly[920]: time="2026-11-02T13:15:19.520Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CSY3QSNBGVTZQ21SDGN7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.506" grpc.time_ms=13.234 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1735gitlab # [ 310.738162] gitaly[920]: time="2026-11-02T13:15:19.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CT432EZR8S714MJ9J88Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.708" grpc.time_ms=10.817 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1736gitlab # [ 310.759856] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1631737gitlab # [ 310.818025] gitaly[920]: time="2026-11-02T13:15:19.799Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CT99STFV29VJ8MHSTG7H grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.781" pid=920 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1738gitlab # [ 310.831836] gitaly[920]: time="2026-11-02T13:15:19.799Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CT99STFV29VJ8MHSTG7H grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.781" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1739gitlab # [ 310.849400] gitlab-workhorse[826]: time="2026-11-02T13:15:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091740gitlab # [ 310.851515] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 901741gitlab # [ 310.853708] gitaly[920]: time="2026-11-02T13:15:19.812Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CT99STFV29VJ8MHSTG7H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:19.781" grpc.time_ms=30.191 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1742gitlab # [ 310.937625] gitaly[920]: time="2026-11-02T13:15:19.919Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:15:49.818" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:15:19.918" grpc.time_ms=0.699 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 311.014296] gitaly[920]: time="2026-11-02T13:15:19.996Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-02T13:15:49.884" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:19.985" grpc.time_ms=11.029 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 311.048364] gitaly[920]: time="2026-11-02T13:15:20.030Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:15:30.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:20.021" grpc.time_ms=8.78 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 311.071258] gitaly[920]: time="2026-11-02T13:15:20.053Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:15:30.021" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:20.041" grpc.time_ms=9.504 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746gitlab # [ 311.171759] gitaly[920]: time="2026-11-02T13:15:20.153Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-02T13:15:30.042" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:20.153" grpc.time_ms=0.347 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1747gitlab # [ 311.191186] gitaly[920]: time="2026-11-02T13:15:20.172Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-02T13:15:30.052" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:20.163" grpc.time_ms=9.446 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1748gitlab # [ 311.234108] gitaly[920]: time="2026-11-02T13:15:20.216Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:15:20.215" grpc.time_ms=0.519 pid=920 span.kind=server system=grpc1749gitlab # [ 311.282619] gitaly[920]: time="2026-11-02T13:15:20.264Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:15:20.263" grpc.time_ms=0.431 pid=920 span.kind=server system=grpc1750gitlab # [ 311.298749] gitaly[920]: time="2026-11-02T13:15:20.276Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 diskcache=c7f6a07f-29f3-4812-9e16-83b8d1b86f77 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-02T13:15:30.065" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:20.176" pid=920 ref=refs/keep-around/1012c4552b1ded9281d4a95af15f7a826db59c66 remote_ip="2001:db8:1::1" revision=1012c4552b1ded9281d4a95af15f7a826db59c66 span.kind=server system=grpc username=root1751gitlab # [ 311.316273] gitaly[920]: time="2026-11-02T13:15:20.276Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=79 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5435 command.oublock=8 command.real_time_ms=94 command.spawn_token_fork_ms=0 command.system_time_ms=59 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-02T13:15:30.065" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:15:20.176" grpc.time_ms=100.219 pid=920 ref=refs/keep-around/1012c4552b1ded9281d4a95af15f7a826db59c66 remote_ip="2001:db8:1::1" revision=1012c4552b1ded9281d4a95af15f7a826db59c66 span.kind=server system=grpc username=root1752gitlab # [ 311.335104] gitaly[920]: time="2026-11-02T13:15:20.280Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:15:50.179" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:15:20.279" grpc.time_ms=0.394 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 311.496835] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1012c4552b1ded9281d4a95af15f7a826db59c66 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6211754gitlab # [ 311.510582] nixbot[3407]: {"ts": "2026-11-02T13:15:20.491Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-1012c4552b1d-0"}1755gitlab # [ 311.711392] nixbot[3407]: {"ts": "2026-11-02T13:15:20.693Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-1012c4552b1d-0", "argv0": "bwrap"}1756gitlab # [ 311.829462] systemd[1]: Started Nix Daemon.1757gitlab # [ 311.910430] nix-daemon[3565]: accepted connection from pid 3560, user nixbot1758gitlab # [ 316.006514] nix-daemon[3565]: accepted connection from pid 3604, user nixbot1759gitlab # [ 316.021752] nix-daemon[3565]: accepted connection from pid 3560, user nixbot1760gitlab # [ 320.206301] nix-daemon[3565]: accepted connection from pid 3560, user nixbot1761gitlab # [ 320.489213] gitaly[920]: time="2026-11-02T13:15:29.470Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D3JXWDNFN749YKZ2MB8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:15:59.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:15:29.469" grpc.time_ms=0.825 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 320.836818] gitaly[920]: time="2026-11-02T13:15:29.818Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6D3JXWDNFN749YKZ2MB8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:15:39.077" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:29.807" grpc.time_ms=10.444 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763gitlab # [ 321.230134] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1012c4552b1ded9281d4a95af15f7a826db59c66 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 9651764gitlab # [ 321.383340] gitaly[920]: time="2026-11-02T13:15:30.365Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D4HJXCENEPPFRXNZ471J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:16:00.063" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:15:30.364" grpc.time_ms=0.657 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1765gitlab # [ 321.451232] gitaly[920]: time="2026-11-02T13:15:30.433Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6D4HJXCENEPPFRXNZ471J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:15:40.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:30.424" grpc.time_ms=8.389 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1766gitlab # [ 321.577835] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1012c4552b1ded9281d4a95af15f7a826db59c66 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3331767gitlab # [ 321.708281] gitaly[920]: time="2026-11-02T13:15:30.689Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D4WC8WRGBBW897ZDDSN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:16:00.287" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:15:30.688" grpc.time_ms=0.754 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768gitlab # [ 321.765426] gitaly[920]: time="2026-11-02T13:15:30.747Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6E6D4WC8WRGBBW897ZDDSN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:15:40.086" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:15:30.737" grpc.time_ms=10.096 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1769gitlab # [ 322.696282] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1012c4552b1ded9281d4a95af15f7a826db59c66 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 11041770gitlab # [ 322.962687] nixbot[3407]: {"ts": "2026-11-02T13:15:31.943Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "1012c4552b1ded9281d4a95af15f7a826db59c66", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1771gitlab # [ 323.081675] gitaly[920]: time="2026-11-02T13:15:32.063Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6E6D68J58ZP6A5X2EG38KZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.052" grpc.time_ms=10.521 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1772gitlab # [ 323.100691] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 761773gitlab # [ 323.185930] gitaly[920]: time="2026-11-02T13:15:32.167Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D6B1B74DXXGM0E6N7M5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.156" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1774gitlab # [ 323.201260] gitaly[920]: time="2026-11-02T13:15:32.167Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D6B1B74DXXGM0E6N7M5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.156" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1775gitlab # [ 323.218459] gitlab-workhorse[826]: time="2026-11-02T13:15:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671776gitlab # [ 323.220721] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1031777gitlab # [ 323.222926] gitaly[920]: time="2026-11-02T13:15:32.168Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D6B1B74DXXGM0E6N7M5P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.156" grpc.time_ms=11.528 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1778gitlab # [ 323.305520] gitaly[920]: time="2026-11-02T13:15:32.287Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6D6FEFE5XD1JYKSRWZDZX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.278" grpc.time_ms=8.645 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1779gitlab # [ 323.322100] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 811780gitlab # [ 323.382168] gitaly[920]: time="2026-11-02T13:15:32.364Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D6J1EW8TPBT44795BJ8P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.353" pid=920 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1781gitlab # [ 323.398738] gitaly[920]: time="2026-11-02T13:15:32.364Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D6J1EW8TPBT44795BJ8P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.353" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1782gitlab # [ 323.420136] gitlab-workhorse[826]: time="2026-11-02T13:15:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671783gitlab # [ 323.422850] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:15:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 751784gitlab # [ 323.426428] gitaly[920]: time="2026-11-02T13:15:32.364Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6D6J1EW8TPBT44795BJ8P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:15:32.353" grpc.time_ms=10.702 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1785gitlab # [ 344.387900] sidekiq-cluster[3287]: 2026-11-02T13:15:53.369Z pid=3287 tid=ygv INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3287, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432259072}1786gitlab # [ 344.672681] sidekiq-cluster[3287]: W, [2026-11-02T13:15:53.654416 #3287] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1787gitlab # [ 344.737348] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:15:53.719Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1788gitlab # [ 348.259959] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:15:57.241Z","message":"GitLab reliable fetch activated!"}1789gitlab # [ 348.260404] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:15:57.241Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1790gitlab # [ 384.917779] postgres[4176]: [4176] ERROR: relation "table_without_models" does not exist1791gitlab # [ 384.918174] postgres[4176]: [4176] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1792gitlab # [ 389.912801] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:38.887Z","message":"Booted Rails 7.2.3.1 application in production environment"}1793gitlab # [ 389.913094] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:38.892Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1794gitlab # [ 389.913162] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:38.892Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1795gitlab # [ 389.913240] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:38.892Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1796gitlab # [ 389.913301] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:38.893Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1797gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 80.77 seconds)1798gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1799gitlab # [ 390.844622] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 741800gitlab # [ 390.937313] gitaly[920]: time="2026-11-02T13:16:39.918Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8GK003E30MW4Q43WY8G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:39.907" grpc.time_ms=10.521 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1801gitlab # [ 390.959476] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 921802gitlab # Enumerating objects: 5, done.1803gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1804gitlab # Delta compression using up to 4 threads1805gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1806gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1807gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1808gitlab # [ 391.277072] gitaly[920]: time="2026-11-02T13:16:40.258Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-02T13:17:10.147" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-02T13:16:40.248" grpc.time_ms=10.251 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1809gitlab # [ 391.326275] gitaly[920]: time="2026-11-02T13:16:40.308Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M6E6F8MVN4NGCZCF31R9A288 duration_ms=188ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/allowed"1810gitlab # [ 391.370385] gitaly[920]: time="2026-11-02T13:16:40.352Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M6E6F8MVN4NGCZCF31R9A288 duration_ms=41ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/pre_receive"1811gitlab # [ 391.376419] gitaly[920]: time="2026-11-02T13:16:40.352Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:40.117" grpc.time_ms=235.473 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1812gitlab # [ 391.421897] gitaly[920]: time="2026-11-02T13:16:40.403Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:40.402" grpc.time_ms=0.942 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1813gitlab # [ 391.485370] gitaly[920]: time="2026-11-02T13:16:40.467Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:40.466" grpc.time_ms=0.53 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1814gitlab # [ 391.544696] gitaly[920]: time="2026-11-02T13:16:40.526Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:40.525" grpc.time_ms=0.699 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1815gitlab # [ 391.722392] gitaly[920]: time="2026-11-02T13:16:40.704Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M6E6F8MVN4NGCZCF31R9A288 duration_ms=127ns method=POST pid=920 status=200 url="http://unix/api/v4/internal/post_receive"1816gitlab # [ 391.728409] gitaly[920]: time="2026-11-02T13:16:40.704Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:40.576" grpc.time_ms=128.68 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1817gitlab # To http://gitlab/root/test-flake.git1818gitlab # [ 391.741496] gitaly[920]: time="2026-11-02T13:16:40.719Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 diskcache=93bbdcbd-9b12-4d1c-af20-c8bb24ef7968 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:40.049" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1819gitlab # 1012c45..5bcc93b [ 391.754324] gitlab-workhorse[826]: time="2026-11-02T13:16:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521820gitlab # master -> master[ 391.756692] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:40 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7561821gitlab # 1822gitlab # [ 391.758977] gitaly[920]: time="2026-11-02T13:16:40.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=255 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13340 command.oublock=32 command.real_time_ms=664 command.spawn_token_fork_ms=0 command.system_time_ms=144 command.user_time_ms=111 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:40.049" grpc.time_ms=669.966 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1823gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.06 seconds)1824gitlab: must succeed: git -C /tmp/test-flake rev-parse master1825gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1826gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'1827gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.10 seconds)1828gitlab # [ 391.888821] nixbot[3407]: {"ts": "2026-11-02T13:16:40.869Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38342 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}1829gitlab # [ 392.631623] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:41.612Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b93bed9bc1c72cb94266cce7","created_at":"2026-11-02T13:15:08.008Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.046Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b93bed9bc1c72cb94266cce7: start","job_status":"start","queue_duration_s":93.565402,"scheduling_latency_s":93.565475}1830gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'1831gitlab # [ 392.844109] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:41.825Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b93bed9bc1c72cb94266cce7","created_at":"2026-11-02T13:15:08.008Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.046Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b93bed9bc1c72cb94266cce7: done: 0.213189 sec","job_status":"done","queue_duration_s":93.565402,"scheduling_latency_s":93.565475,"redis_calls":23,"redis_duration_s":0.00534,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002609,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001186,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.1056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11368166,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.213189,"completed_at":"2026-11-02T13:16:41.825Z","load_balancing_strategy":"primary","db_duration_s":0.063016,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1832gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.04 seconds)1833gitlab # [ 392.915226] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:41.827Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"483b94604ec10932ab4f6d30","created_at":"2026-11-02T13:15:08.059Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.065Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-483b94604ec10932ab4f6d30: start","job_status":"start","queue_duration_s":93.762087,"scheduling_latency_s":93.762114}1834gitlab # [ 392.936487] nixbot[3407]: {"ts": "2026-11-02T13:16:41.906Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38356 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}1835gitlab # [ 392.980096] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:41.961Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"483b94604ec10932ab4f6d30","created_at":"2026-11-02T13:15:08.059Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.065Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-483b94604ec10932ab4f6d30: done: 0.134366 sec","job_status":"done","queue_duration_s":93.762087,"scheduling_latency_s":93.762114,"redis_calls":14,"redis_duration_s":0.003388,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000624,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0018,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10430584000000001,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.134366,"completed_at":"2026-11-02T13:16:41.961Z","load_balancing_strategy":"primary","db_duration_s":0.023377,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1836gitlab # [ 393.043921] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:41.963Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1cf9cee981bbe46d72a44ab6","created_at":"2026-11-02T13:15:08.556Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.562Z","job_size_bytes":5,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1cf9cee981bbe46d72a44ab6: start","job_status":"start","queue_duration_s":93.400658,"scheduling_latency_s":93.400677}1837gitlab # [ 393.154290] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:42.135Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1cf9cee981bbe46d72a44ab6","created_at":"2026-11-02T13:15:08.556Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.562Z","job_size_bytes":5,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1cf9cee981bbe46d72a44ab6: done: 0.172682 sec","job_status":"done","queue_duration_s":93.400658,"scheduling_latency_s":93.400677,"redis_calls":12,"redis_duration_s":0.00222,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00134,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027072,"duration_s":0.172682,"completed_at":"2026-11-02T13:16:42.135Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024847,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1838gitlab # [ 393.223281] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:42.137Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"efe0198bf70c55259c9dce06","created_at":"2026-11-02T13:15:08.590Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.594Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-efe0198bf70c55259c9dce06: start","job_status":"start","queue_duration_s":93.542302,"scheduling_latency_s":93.54232}1839gitlab # [ 393.390469] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:42.371Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"efe0198bf70c55259c9dce06","created_at":"2026-11-02T13:15:08.590Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.594Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-efe0198bf70c55259c9dce06: done: 0.234694 sec","job_status":"done","queue_duration_s":93.542302,"scheduling_latency_s":93.54232,"redis_calls":11,"redis_duration_s":0.002087,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.197,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.197,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.17337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032954,"duration_s":0.234694,"completed_at":"2026-11-02T13:16:42.371Z","load_balancing_strategy":"primary","db_duration_s":0.065039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1840gitlab # [ 393.457963] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:42.373Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f758b95ea5954ce881a71ebe","created_at":"2026-11-02T13:15:08.598Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.602Z","job_size_bytes":89,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f758b95ea5954ce881a71ebe: start","job_status":"start","queue_duration_s":93.770844,"scheduling_latency_s":93.770868}1841gitlab # [ 393.478524] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:42.415Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f758b95ea5954ce881a71ebe","created_at":"2026-11-02T13:15:08.598Z","trace_propagation_headers":{"sentry-trace":"a871c3d21d804bd8815efcbfaefe745f-79e992bbee1941ca","baggage":"sentry-trace_id=a871c3d21d804bd8815efcbfaefe745f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CDXEPZ27XA2SEY0RY79R","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:08.602Z","job_size_bytes":89,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f758b95ea5954ce881a71ebe: done: 0.041871 sec","job_status":"done","queue_duration_s":93.770844,"scheduling_latency_s":93.770868,"redis_calls":11,"redis_duration_s":0.001999,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000294,"duration_s":0.041871,"completed_at":"2026-11-02T13:16:42.415Z","load_balancing_strategy":"primary","db_duration_s":0.005411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1842gitlab # [ 393.526885] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:42.416Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 1012c4552b1ded9281d4a95af15f7a826db59c66 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"116bd79fd0898e4463d0bc7c","created_at":"2026-11-02T13:15:15.445Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-baa7e443a8c64a6b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:15:15.474Z","job_size_bytes":185,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-116bd79fd0898e4463d0bc7c: start","job_status":"start","queue_duration_s":86.942157,"scheduling_latency_s":86.942175}1843gitlab # [ 393.544695] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:42.433Z","message":"changes: \"0000000000000000000000000000000000000000 1012c4552b1ded9281d4a95af15f7a826db59c66 refs/heads/master\\n\""}1844gitlab # [ 393.547569] gitaly[920]: time="2026-11-02T13:16:42.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:52.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:42.460" grpc.time_ms=10.458 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 393.561029] gitaly[920]: time="2026-11-02T13:16:42.485Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:16:52.015" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:42.476" grpc.time_ms=8.839 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846gitlab # [ 393.574734] gitaly[920]: time="2026-11-02T13:16:42.539Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-02T13:16:52.067" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:42.528" grpc.time_ms=11.388 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1847gitlab # [ 393.602774] gitaly[920]: time="2026-11-02T13:16:42.584Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:42.584" grpc.time_ms=0.453 pid=920 span.kind=server system=grpc1848gitlab # [ 393.632812] gitaly[920]: time="2026-11-02T13:16:42.614Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:42.614" grpc.time_ms=0.457 pid=920 span.kind=server system=grpc1849gitlab # [ 393.645917] gitaly[920]: time="2026-11-02T13:16:42.624Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N diskcache=79b15958-1470-46a7-962e-5c28dc7b919e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-02T13:16:52.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:42.553" pid=920 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1850gitlab # [ 393.659841] gitaly[920]: time="2026-11-02T13:16:42.624Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4887 command.oublock=8 command.real_time_ms=67 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-02T13:16:52.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:42.553" grpc.time_ms=71.678 pid=920 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1851gitlab # [ 393.675584] gitaly[920]: time="2026-11-02T13:16:42.640Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-02T13:16:52.180" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:42.640" grpc.time_ms=0.185 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1852gitlab # [ 393.686795] gitaly[920]: time="2026-11-02T13:16:42.660Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=860 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-02T13:16:52.186" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:42.646" grpc.time_ms=13.741 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853gitlab # [ 393.790880] gitaly[920]: time="2026-11-02T13:16:42.772Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-02T13:17:12.064" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:42.765" grpc.time_ms=7.449 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1854gitlab # [ 393.828541] gitaly[920]: time="2026-11-02T13:16:42.810Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-02T13:16:52.009" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:42.809" grpc.time_ms=0.421 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1855gitlab # [ 393.844179] gitaly[920]: time="2026-11-02T13:16:42.819Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-02T13:16:52.012" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-02T13:16:42.813" grpc.time_ms=6.667 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856gitlab # [ 393.864887] gitaly[920]: time="2026-11-02T13:16:42.846Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:52.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:42.836" grpc.time_ms=10.023 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857gitlab # [ 393.881333] gitaly[920]: time="2026-11-02T13:16:42.863Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:52.052" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:42.853" grpc.time_ms=9.833 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'1859gitlab # [ 393.912781] gitaly[920]: time="2026-11-02T13:16:42.894Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-02T13:17:12.177" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:42.877" grpc.time_ms=16.973 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.04 seconds)1861gitlab # [ 393.966386] nixbot[3407]: {"ts": "2026-11-02T13:16:42.947Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38360 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}1862gitlab # [ 394.410527] gitaly[920]: time="2026-11-02T13:16:43.392Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-02T13:16:53.010" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:43.390" grpc.time_ms=1.77 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863gitlab # [ 394.441798] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:43.422Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 1012c4552b1ded9281d4a95af15f7a826db59c66 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"116bd79fd0898e4463d0bc7c","created_at":"2026-11-02T13:15:15.445Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-baa7e443a8c64a6b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:15:15.474Z","job_size_bytes":185,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-116bd79fd0898e4463d0bc7c: done: 1.005519 sec","job_status":"done","queue_duration_s":86.942157,"scheduling_latency_s":86.942175,"gitaly_calls":13,"gitaly_duration_s":0.207031,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.040293,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.003644,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.0052,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001523,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003424,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.01744,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.009062,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.268,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.298,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.719946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00061388,"exclusive_lock_hold_duration_s":0.02788916,"duration_s":1.005519,"completed_at":"2026-11-02T13:16:43.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.08779,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1864gitlab # [ 394.513337] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:43.424Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"66314258fd8b0385f6acc771","created_at":"2026-11-02T13:15:20.293Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-2fa4c088c2704802","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:20.302Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-66314258fd8b0385f6acc771: start","job_status":"start","queue_duration_s":83.121936,"scheduling_latency_s":83.121957}1865gitlab # [ 394.531264] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:43.506Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"66314258fd8b0385f6acc771","created_at":"2026-11-02T13:15:20.293Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-2fa4c088c2704802","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:20.302Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-66314258fd8b0385f6acc771: done: 0.08289 sec","job_status":"done","queue_duration_s":83.121936,"scheduling_latency_s":83.121957,"redis_calls":16,"redis_duration_s":0.003068,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001423,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000391,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043172,"duration_s":0.08289,"completed_at":"2026-11-02T13:16:43.506Z","load_balancing_strategy":"primary","db_duration_s":0.019681,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1866gitlab # [ 394.578977] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:43.508Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"97f4a14a5e42d95f375691fe","created_at":"2026-11-02T13:15:20.303Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-2fa4c088c2704802","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:20.309Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-97f4a14a5e42d95f375691fe: start","job_status":"start","queue_duration_s":83.199311,"scheduling_latency_s":83.199329}1867gitlab # [ 394.760175] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:43.741Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"97f4a14a5e42d95f375691fe","created_at":"2026-11-02T13:15:20.303Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-2fa4c088c2704802","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:20.309Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-97f4a14a5e42d95f375691fe: done: 0.232945 sec","job_status":"done","queue_duration_s":83.199311,"scheduling_latency_s":83.199329,"redis_calls":21,"redis_duration_s":0.0059889999999999995,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002071,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00253,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.192433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031726,"duration_s":0.232945,"completed_at":"2026-11-02T13:16:43.741Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1868gitlab # [ 394.830924] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:43.742Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2ee87bc1aa279addbec4c678","created_at":"2026-11-02T13:15:20.452Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-2fa4c088c2704802","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:20.458Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2ee87bc1aa279addbec4c678: start","job_status":"start","queue_duration_s":83.283976,"scheduling_latency_s":83.283994}1869gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'1870gitlab # [ 394.977249] gitaly[920]: time="2026-11-02T13:16:43.958Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:43.957" grpc.time_ms=0.673 pid=920 span.kind=server system=grpc1871gitlab # [ 395.003172] nixbot[3407]: {"ts": "2026-11-02T13:16:43.984Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38370 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}1872gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.04 seconds)1873gitlab # [ 395.014575] gitaly[920]: time="2026-11-02T13:16:43.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:43.995" grpc.time_ms=0.698 pid=920 span.kind=server system=grpc1874gitlab # [ 395.043317] gitaly[920]: time="2026-11-02T13:16:44.025Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:44.024" grpc.time_ms=0.535 pid=920 span.kind=server system=grpc1875gitlab # [ 395.060408] gitaly[920]: time="2026-11-02T13:16:44.032Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 diskcache=6fd83bb6-b2e8-41b5-bc03-5deef08642d7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:13.224" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:43.925" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 395.076166] gitaly[920]: time="2026-11-02T13:16:44.032Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=112 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7408 command.oublock=0 command.real_time_ms=102 command.spawn_token_fork_ms=0 command.system_time_ms=55 command.user_time_ms=57 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:13.224" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:43.925" grpc.time_ms=107.359 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877gitlab # [ 395.196361] gitaly[920]: time="2026-11-02T13:16:44.178Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-02T13:17:14.076" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.177" grpc.time_ms=1.105 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 395.247040] gitaly[920]: time="2026-11-02T13:16:44.228Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CTDGQEM078MPEGKN4213 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:14.127" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.227" grpc.time_ms=0.9 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 395.272798] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.253Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2ee87bc1aa279addbec4c678","created_at":"2026-11-02T13:15:20.452Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-2fa4c088c2704802","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-02T13:15:20.458Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2ee87bc1aa279addbec4c678: done: 0.510986 sec","job_status":"done","queue_duration_s":83.283976,"scheduling_latency_s":83.283994,"gitaly_calls":3,"gitaly_duration_s":0.11769,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.026048,"redis_read_bytes":455,"redis_write_bytes":27998,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001679,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001409,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008904,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.003488,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14492,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.005481,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000902,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000504,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002002,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.029,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.033,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.313089,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.44088066,"duration_s":0.510986,"completed_at":"2026-11-02T13:16:44.253Z","load_balancing_strategy":"primary","db_duration_s":0.074291,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1880gitlab # [ 395.347147] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.255Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","1012c4552b1ded9281d4a95af15f7a826db59c66 5bcc93bc627dd5c1ed48f56f948c7335c5310516 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f6d9b4504c2c677e46093077","created_at":"2026-11-02T13:16:40.668Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-239e8370ebdf445c","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:40.672Z","job_size_bytes":185,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f6d9b4504c2c677e46093077: start","job_status":"start","queue_duration_s":3.583295,"scheduling_latency_s":3.58332}1881gitlab # [ 395.364899] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.269Z","message":"changes: \"1012c4552b1ded9281d4a95af15f7a826db59c66 5bcc93bc627dd5c1ed48f56f948c7335c5310516 refs/heads/master\\n\""}1882gitlab # [ 395.367380] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.312Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"9d29a943030de4b31da93eb9","created_at":"2026-11-02T13:16:44.311Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-eaa2354c1a57415a","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"122960f1238153d232ed9b22","job_size_bytes":46,"pid":3287,"job_status":"deduplicated","message":"ProjectCacheWorker JID-9d29a943030de4b31da93eb9: deduplicated: until executing","deduplication.type":"until executing"}1883gitlab # [ 395.383688] gitaly[920]: time="2026-11-02T13:16:44.295Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:54.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:44.285" grpc.time_ms=9.929 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 395.397209] gitaly[920]: time="2026-11-02T13:16:44.308Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-02T13:16:54.020" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:44.300" grpc.time_ms=8.327 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 395.410921] gitaly[920]: time="2026-11-02T13:16:44.333Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-02T13:16:54.044" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:44.324" grpc.time_ms=9.103 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886gitlab # [ 395.424545] gitaly[920]: time="2026-11-02T13:16:44.347Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-02T13:17:14.237" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.337" grpc.time_ms=10.335 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1887gitlab # [ 395.440530] gitaly[920]: time="2026-11-02T13:16:44.361Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-02T13:16:54.077" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-02T13:16:44.357" grpc.time_ms=4.475 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab # [ 395.454101] gitaly[920]: time="2026-11-02T13:16:44.378Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:54.089" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:44.369" grpc.time_ms=9.167 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab # [ 395.497965] gitaly[920]: time="2026-11-02T13:16:44.479Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-02T13:16:54.198" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.478" grpc.time_ms=1.237 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 395.525413] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.506Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","1012c4552b1ded9281d4a95af15f7a826db59c66 5bcc93bc627dd5c1ed48f56f948c7335c5310516 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f6d9b4504c2c677e46093077","created_at":"2026-11-02T13:16:40.668Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-239e8370ebdf445c","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:40.672Z","job_size_bytes":185,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f6d9b4504c2c677e46093077: done: 0.250189 sec","job_status":"done","queue_duration_s":3.583295,"scheduling_latency_s":3.58332,"gitaly_calls":7,"gitaly_duration_s":0.068008,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.016704,"redis_read_bytes":501,"redis_write_bytes":11940,"redis_cache_calls":9,"redis_cache_duration_s":0.001655,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001223,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5717,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002744,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.005263,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.005819,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.156387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00061616,"exclusive_lock_hold_duration_s":0.01839668,"duration_s":0.250189,"completed_at":"2026-11-02T13:16:44.505Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025873,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1891gitlab # [ 395.593237] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.507Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"122960f1238153d232ed9b22","created_at":"2026-11-02T13:16:42.487Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:42.504Z","job_size_bytes":46,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-122960f1238153d232ed9b22: start","job_status":"start","queue_duration_s":2.003299,"scheduling_latency_s":2.003325}1892gitlab # [ 395.610403] gitaly[920]: time="2026-11-02T13:16:44.572Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-02T19:16:44.566" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:44.566" grpc.time_ms=5.23 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893gitlab # [ 395.622097] gitaly[920]: time="2026-11-02T13:16:44.590Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-02T13:17:14.480" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.581" grpc.time_ms=9.448 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894gitlab # [ 395.638122] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.619Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"122960f1238153d232ed9b22","created_at":"2026-11-02T13:16:42.487Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:42.504Z","job_size_bytes":46,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-122960f1238153d232ed9b22: done: 0.111725 sec","job_status":"done","queue_duration_s":2.003299,"scheduling_latency_s":2.003325,"gitaly_calls":2,"gitaly_duration_s":0.019181,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0064069999999999995,"redis_read_bytes":69,"redis_write_bytes":4882,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001375,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000517,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002433,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003052,"duration_s":0.111725,"completed_at":"2026-11-02T13:16:44.619Z","load_balancing_strategy":"primary","db_duration_s":0.010717,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1895gitlab # [ 395.690165] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.621Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"943eb3e09fa57578860a0835","created_at":"2026-11-02T13:16:42.520Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:42.524Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-943eb3e09fa57578860a0835: start","job_status":"start","queue_duration_s":2.096341,"scheduling_latency_s":2.096359}1896gitlab # [ 395.707347] gitaly[920]: time="2026-11-02T13:16:44.667Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N diskcache=4738db5f-b179-444b-9368-0e89a12a828f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-02T19:16:44.645" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.645" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab # [ 395.717676] gitaly[920]: time="2026-11-02T13:16:44.667Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-02T19:16:44.645" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.645" grpc.time_ms=21.598 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 395.734098] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.712Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"943eb3e09fa57578860a0835","created_at":"2026-11-02T13:16:42.520Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:42.524Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-943eb3e09fa57578860a0835: done: 0.091431 sec","job_status":"done","queue_duration_s":2.096341,"scheduling_latency_s":2.096359,"gitaly_calls":1,"gitaly_duration_s":0.023761,"redis_calls":15,"redis_duration_s":0.003294,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001277,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000469,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06352105999999999,"duration_s":0.091431,"completed_at":"2026-11-02T13:16:44.712Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1899gitlab # [ 395.782431] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.714Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"cbcf4eb594276a856d87736a","created_at":"2026-11-02T13:16:42.635Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:42.637Z","job_size_bytes":23,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-cbcf4eb594276a856d87736a: start","job_status":"start","queue_duration_s":2.076943,"scheduling_latency_s":2.07697}1900gitlab # [ 395.798508] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.775Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"cbcf4eb594276a856d87736a","created_at":"2026-11-02T13:16:42.635Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:42.637Z","job_size_bytes":23,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-cbcf4eb594276a856d87736a: done: 0.060765 sec","job_status":"done","queue_duration_s":2.076943,"scheduling_latency_s":2.07697,"gitaly_calls":1,"gitaly_duration_s":0.012815,"redis_calls":16,"redis_duration_s":0.003548,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000302,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001543,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.060765,"completed_at":"2026-11-02T13:16:44.775Z","load_balancing_strategy":"primary","db_duration_s":0.003624,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1901gitlab # [ 395.845827] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:44.777Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b43bd5e250279253886908c3","created_at":"2026-11-02T13:16:43.218Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:51f2cdc2ddd024f959936d824aa1eef97e55c074b544569a8c7afea72dc0ba81","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:43.224Z","job_size_bytes":467,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b43bd5e250279253886908c3: start","job_status":"start","queue_duration_s":1.552613,"scheduling_latency_s":1.55264}1902gitlab # [ 395.863600] gitaly[920]: time="2026-11-02T13:16:44.762Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-02T13:17:14.651" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.751" grpc.time_ms=10.087 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903gitlab # [ 395.877291] gitaly[920]: time="2026-11-02T13:16:44.816Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:14.714" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.814" grpc.time_ms=0.948 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 395.889853] gitaly[920]: time="2026-11-02T13:16:44.829Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-02T13:16:54.018" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:44.819" grpc.time_ms=10.127 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 395.987698] gitaly[920]: time="2026-11-02T13:16:44.969Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-02T13:17:14.867" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.967" grpc.time_ms=1.008 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'1907gitlab # [ 396.008339] gitaly[920]: time="2026-11-02T13:16:44.975Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=34 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-02T13:17:14.872" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:44.973" grpc.time_ms=2.696 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.05 seconds)1909gitlab # [ 396.051580] nixbot[3407]: {"ts": "2026-11-02T13:16:45.032Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38386 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}1910gitlab # [ 396.099268] gitaly[920]: time="2026-11-02T13:16:45.081Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:55.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:45.071" grpc.time_ms=9.932 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab # [ 396.677682] gitaly[920]: time="2026-11-02T13:16:45.659Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=879 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-02T13:17:15.542" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:45.642" grpc.time_ms=16.488 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 396.702194] gitaly[920]: time="2026-11-02T13:16:45.678Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6E6CNGKPXB9YVQ20KNKXK9N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-02T13:17:15.562" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:45.662" grpc.time_ms=15.49 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 396.726400] sidekiq-cluster[3287]: {"severity":"WARN","time":"2026-11-02T13:16:45.707Z","class":"CreatePipelineWorker","correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"1012c4552b1ded9281d4a95af15f7a826db59c66","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"1012c4552b1ded9281d4a95af15f7a826db59c66","ref":"refs/heads/master"},"retry":0}1914gitlab # [ 396.737401] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:45.715Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b43bd5e250279253886908c3","created_at":"2026-11-02T13:16:43.218Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:51f2cdc2ddd024f959936d824aa1eef97e55c074b544569a8c7afea72dc0ba81","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:43.224Z","job_size_bytes":467,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b43bd5e250279253886908c3: done: 0.93824 sec","job_status":"done","queue_duration_s":1.552613,"scheduling_latency_s":1.55264,"gitaly_calls":7,"gitaly_duration_s":0.074373,"redis_calls":51,"redis_duration_s":0.009112,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000924,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003782,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002354,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.810657,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002795,"duration_s":0.93824,"completed_at":"2026-11-02T13:16:45.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059579,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1915gitlab # [ 396.799488] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:45.717Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"85051330-bfdc-4bac-be29-2df68ff58e34\"}"],"class":"WebHookWorker","jid":"7ea342827de65d55628dd3e6","created_at":"2026-11-02T13:16:43.358Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8923a2a50f48408f07703259ff0e8c5d0aea5ca0208ecbec273a5115f37f3516","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:43.364Z","job_size_bytes":1751,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7ea342827de65d55628dd3e6: start","job_status":"start","queue_duration_s":2.353003,"scheduling_latency_s":2.353025}1916gitlab # [ 396.883015] nixbot[3407]: {"ts": "2026-11-02T13:16:45.864Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "5ac26a8d-6a25-4ffb-9a9c-5958320d23c5", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "1012c4552b1ded9281d4a95af15f7a826db59c66", "pr": null}1917gitlab # [ 396.889174] nixbot[3407]: {"ts": "2026-11-02T13:16:45.870Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38402 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1918gitlab # [ 396.914379] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:45.895Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"85051330-bfdc-4bac-be29-2df68ff58e34\"}"],"class":"WebHookWorker","jid":"7ea342827de65d55628dd3e6","created_at":"2026-11-02T13:16:43.358Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-ac113ee602f5443b","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8923a2a50f48408f07703259ff0e8c5d0aea5ca0208ecbec273a5115f37f3516","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:43.364Z","job_size_bytes":1751,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7ea342827de65d55628dd3e6: done: 0.178381 sec","job_status":"done","queue_duration_s":2.353003,"scheduling_latency_s":2.353025,"redis_calls":19,"redis_duration_s":0.004423,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001122,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000504,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002019,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000778,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07121874000000616,"cpu_s":0.093645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036178,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.178381,"completed_at":"2026-11-02T13:16:45.895Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1919gitlab # [ 396.984572] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:45.897Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2f5e6f3866956d6c0e046b29","created_at":"2026-11-02T13:16:43.904Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:43.910Z","job_size_bytes":23,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2f5e6f3866956d6c0e046b29: start","job_status":"start","queue_duration_s":1.987178,"scheduling_latency_s":1.987209}1920gitlab # [ 397.002677] gitaly[920]: time="2026-11-02T13:16:45.976Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FEEBN4ECA655YD4SZBDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:45.962" grpc.time_ms=13.428 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1921gitlab # [ 397.016749] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 771922gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'1923gitlab # [ 397.024696] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.006Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2f5e6f3866956d6c0e046b29","created_at":"2026-11-02T13:16:43.904Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:43.910Z","job_size_bytes":23,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2f5e6f3866956d6c0e046b29: done: 0.108316 sec","job_status":"done","queue_duration_s":1.987178,"scheduling_latency_s":1.987209,"redis_calls":10,"redis_duration_s":0.002443,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000408,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034384,"duration_s":0.108316,"completed_at":"2026-11-02T13:16:46.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017651,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1924gitlab # [ 397.074732] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.007Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"120b435476c8243a605fdecf","created_at":"2026-11-02T13:16:44.043Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.050Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-120b435476c8243a605fdecf: start","job_status":"start","queue_duration_s":1.956809,"scheduling_latency_s":1.95683}1925gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.04 seconds)1926gitlab # [ 397.092224] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.053Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"120b435476c8243a605fdecf","created_at":"2026-11-02T13:16:44.043Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.050Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-120b435476c8243a605fdecf: done: 0.046224 sec","job_status":"done","queue_duration_s":1.956809,"scheduling_latency_s":1.95683,"redis_calls":10,"redis_duration_s":0.002682,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000363,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00067862,"duration_s":0.046224,"completed_at":"2026-11-02T13:16:46.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1927gitlab # [ 397.137699] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.059Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6d2afdd9135220357b38333b","created_at":"2026-11-02T13:16:44.068Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.077Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6d2afdd9135220357b38333b: start","job_status":"start","queue_duration_s":1.98213,"scheduling_latency_s":1.98215}1928gitlab # [ 397.153922] gitlab-workhorse[826]: time="2026-11-02T13:16:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671929gitlab # [ 397.155797] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 931930gitlab # [ 397.158207] nixbot[3407]: {"ts": "2026-11-02T13:16:46.067Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38418 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}1931gitlab # [ 397.161132] gitaly[920]: time="2026-11-02T13:16:46.071Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEGTC3SJB7QT9D41QH6Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:46.062" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1932gitlab # [ 397.172351] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.103Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6d2afdd9135220357b38333b","created_at":"2026-11-02T13:16:44.068Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.077Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6d2afdd9135220357b38333b: done: 0.044517 sec","job_status":"done","queue_duration_s":1.98213,"scheduling_latency_s":1.98215,"redis_calls":10,"redis_duration_s":0.001892,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000369,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027416,"duration_s":0.044517,"completed_at":"2026-11-02T13:16:46.103Z","load_balancing_strategy":"primary","db_duration_s":0.008806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1933gitlab # [ 397.216902] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.105Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b29378926d9928de175c7ef9","created_at":"2026-11-02T13:16:44.095Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.102Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b29378926d9928de175c7ef9: start","job_status":"start","queue_duration_s":2.002644,"scheduling_latency_s":2.002662}1934gitlab # [ 397.233837] gitlab-workhorse[826]: time="2026-11-02T13:16:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371935gitlab # [ 397.235709] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 711936gitlab # [ 397.237797] gitaly[920]: time="2026-11-02T13:16:46.071Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEGTC3SJB7QT9D41QH6Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:46.062" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1937gitlab # [ 397.251564] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.149Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b29378926d9928de175c7ef9","created_at":"2026-11-02T13:16:44.095Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.102Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b29378926d9928de175c7ef9: done: 0.044071 sec","job_status":"done","queue_duration_s":2.002644,"scheduling_latency_s":2.002662,"redis_calls":13,"redis_duration_s":0.0032500000000000003,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000358,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000573,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00066,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031724,"duration_s":0.044071,"completed_at":"2026-11-02T13:16:46.149Z","load_balancing_strategy":"primary","db_duration_s":0.006797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1938gitlab # [ 397.298214] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.151Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f0cb41ffafd8f7afe03dc1ed","created_at":"2026-11-02T13:16:44.103Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.108Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f0cb41ffafd8f7afe03dc1ed: start","job_status":"start","queue_duration_s":2.042307,"scheduling_latency_s":2.042329}1939gitlab # [ 397.315086] nixbot[3407]: {"ts": "2026-11-02T13:16:46.178Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1940gitlab # [ 397.317702] nixbot[3407]: {"ts": "2026-11-02T13:16:46.180Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "1012c4552b1ded9281d4a95af15f7a826db59c66"}1941gitlab # [ 397.320272] gitaly[920]: time="2026-11-02T13:16:46.071Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEGTC3SJB7QT9D41QH6Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:46.062" grpc.time_ms=8.626 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1942gitlab # [ 397.334944] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f0cb41ffafd8f7afe03dc1ed","created_at":"2026-11-02T13:16:44.103Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.108Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f0cb41ffafd8f7afe03dc1ed: done: 0.039924 sec","job_status":"done","queue_duration_s":2.042307,"scheduling_latency_s":2.042329,"redis_calls":10,"redis_duration_s":0.0032890000000000003,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000818,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00085546,"duration_s":0.039924,"completed_at":"2026-11-02T13:16:46.191Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005859,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1943gitlab # [ 397.380689] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.192Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7ee265d2175f0975e5cc497c","created_at":"2026-11-02T13:16:44.109Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.115Z","job_size_bytes":19,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7ee265d2175f0975e5cc497c: start","job_status":"start","queue_duration_s":2.077154,"scheduling_latency_s":2.077174}1944gitlab # [ 397.397246] gitaly[920]: time="2026-11-02T13:16:46.144Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEKT1N5VC4HVPZVWR49P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:46.127" pid=920 remote_ip="2001:db8:1::1" request_sha=dbd206136393861f9c9274ed2015e03b2316b4da response_bytes=337 span.kind=server system=grpc1945gitlab # [ 397.407696] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.226Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7ee265d2175f0975e5cc497c","created_at":"2026-11-02T13:16:44.109Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.115Z","job_size_bytes":19,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7ee265d2175f0975e5cc497c: done: 0.034139 sec","job_status":"done","queue_duration_s":2.077154,"scheduling_latency_s":2.077174,"redis_calls":11,"redis_duration_s":0.0027,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001853,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003501,"duration_s":0.034139,"completed_at":"2026-11-02T13:16:46.226Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004746,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1946gitlab # [ 397.454213] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.228Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"786a682baf01d9bf0b348cfb","created_at":"2026-11-02T13:16:44.118Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.124Z","job_size_bytes":66,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-786a682baf01d9bf0b348cfb: start","job_status":"start","queue_duration_s":2.103737,"scheduling_latency_s":2.103756}1947gitlab # [ 397.472145] gitaly[920]: time="2026-11-02T13:16:46.144Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEKT1N5VC4HVPZVWR49P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:46.127" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1948gitlab # [ 397.486118] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.264Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"786a682baf01d9bf0b348cfb","created_at":"2026-11-02T13:16:44.118Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.124Z","job_size_bytes":66,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-786a682baf01d9bf0b348cfb: done: 0.036383 sec","job_status":"done","queue_duration_s":2.103737,"scheduling_latency_s":2.103756,"redis_calls":10,"redis_duration_s":0.001985,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000367,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027768,"duration_s":0.036383,"completed_at":"2026-11-02T13:16:46.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006805,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1949gitlab # [ 397.532218] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.266Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1e1fd266a80fec7030248f56","created_at":"2026-11-02T13:16:44.125Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.130Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1e1fd266a80fec7030248f56: start","job_status":"start","queue_duration_s":2.136627,"scheduling_latency_s":2.136646}1950gitlab # [ 397.548653] gitaly[920]: time="2026-11-02T13:16:46.145Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEKT1N5VC4HVPZVWR49P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:46.127" grpc.time_ms=17.113 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1951gitlab # [ 397.563655] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1012c4552b1ded9281d4a95af15f7a826db59c66 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2781952gitlab # [ 397.566409] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.327Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1e1fd266a80fec7030248f56","created_at":"2026-11-02T13:16:44.125Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.130Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1e1fd266a80fec7030248f56: done: 0.060151 sec","job_status":"done","queue_duration_s":2.136627,"scheduling_latency_s":2.136646,"redis_calls":10,"redis_duration_s":0.003127,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000356,"duration_s":0.060151,"completed_at":"2026-11-02T13:16:46.326Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 397.611347] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.328Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"eaa25e504e68cc863ddb619b","created_at":"2026-11-02T13:16:44.137Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.142Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-eaa25e504e68cc863ddb619b: start","job_status":"start","queue_duration_s":2.185889,"scheduling_latency_s":2.185913}1954gitlab # [ 397.628594] gitaly[920]: time="2026-11-02T13:16:46.251Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:16.150" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.250" grpc.time_ms=0.755 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1955gitlab # [ 397.642696] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.372Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"eaa25e504e68cc863ddb619b","created_at":"2026-11-02T13:16:44.137Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.142Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-eaa25e504e68cc863ddb619b: done: 0.043683 sec","job_status":"done","queue_duration_s":2.185889,"scheduling_latency_s":2.185913,"redis_calls":10,"redis_duration_s":0.001757,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00033,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000353,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027028,"duration_s":0.043683,"completed_at":"2026-11-02T13:16:46.372Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1956gitlab # [ 397.688650] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.374Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3f2d6f94353bd523630a77d0","created_at":"2026-11-02T13:16:44.143Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.148Z","job_size_bytes":6,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3f2d6f94353bd523630a77d0: start","job_status":"start","queue_duration_s":2.225962,"scheduling_latency_s":2.225984}1957gitlab # [ 397.704809] gitaly[920]: time="2026-11-02T13:16:46.410Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:56.286" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:46.396" grpc.time_ms=13.489 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958gitlab # [ 397.719169] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1012c4552b1ded9281d4a95af15f7a826db59c66 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1951959gitlab # [ 397.721919] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.400Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3f2d6f94353bd523630a77d0","created_at":"2026-11-02T13:16:44.143Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.148Z","job_size_bytes":6,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3f2d6f94353bd523630a77d0: done: 0.026801 sec","job_status":"done","queue_duration_s":2.225962,"scheduling_latency_s":2.225984,"redis_calls":10,"redis_duration_s":0.00164,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000384,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029392,"duration_s":0.026801,"completed_at":"2026-11-02T13:16:46.400Z","load_balancing_strategy":"primary","db_duration_s":0.0024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1960gitlab # [ 397.767212] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.402Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"65a7ec76c05b869f74d69874","created_at":"2026-11-02T13:16:44.148Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.153Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-65a7ec76c05b869f74d69874: start","job_status":"start","queue_duration_s":2.249001,"scheduling_latency_s":2.249022}1961gitlab # [ 397.783088] gitaly[920]: time="2026-11-02T13:16:46.556Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FF0KPQ673W5E0NS5R9P2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:16.455" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.555" grpc.time_ms=0.624 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 397.796352] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.590Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"65a7ec76c05b869f74d69874","created_at":"2026-11-02T13:16:44.148Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.153Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-65a7ec76c05b869f74d69874: done: 0.188625 sec","job_status":"done","queue_duration_s":2.249001,"scheduling_latency_s":2.249022,"redis_calls":10,"redis_duration_s":0.001776,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000315,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.167233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027024,"duration_s":0.188625,"completed_at":"2026-11-02T13:16:46.590Z","load_balancing_strategy":"primary","db_duration_s":0.008881,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1963gitlab # [ 397.841630] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.592Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"9055816fd10e08f2b8f43d98","created_at":"2026-11-02T13:16:44.159Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.165Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-9055816fd10e08f2b8f43d98: start","job_status":"start","queue_duration_s":2.426866,"scheduling_latency_s":2.426887}1964gitlab # [ 397.858656] gitaly[920]: time="2026-11-02T13:16:46.615Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FF0KPQ673W5E0NS5R9P2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:56.101" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:46.602" grpc.time_ms=12.819 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1965gitlab # [ 397.872603] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.668Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"9055816fd10e08f2b8f43d98","created_at":"2026-11-02T13:16:44.159Z","trace_propagation_headers":{"sentry-trace":"a3f8822097bc4a4b9e2b5c1669d906a6-e360688c693443b5","baggage":"sentry-trace_id=a3f8822097bc4a4b9e2b5c1669d906a6,sentry-release=Unknown"},"correlation_id":"01M6E6CTDGQEM078MPEGKN4213","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.165Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-9055816fd10e08f2b8f43d98: done: 0.07587 sec","job_status":"done","queue_duration_s":2.426866,"scheduling_latency_s":2.426887,"redis_calls":14,"redis_duration_s":0.0039299999999999995,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039106,"duration_s":0.07587,"completed_at":"2026-11-02T13:16:46.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1966gitlab # [ 397.918448] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.669Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c1cc99a97704a0ec5bb92fd9","created_at":"2026-11-02T13:16:44.318Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-eaa2354c1a57415a","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"943eb3e09fa57578860a0835","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.321Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c1cc99a97704a0ec5bb92fd9: start","job_status":"start","queue_duration_s":2.348398,"scheduling_latency_s":2.348417}1967gitlab # [ 397.935970] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1012c4552b1ded9281d4a95af15f7a826db59c66 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1601968gitlab # [ 397.938756] gitaly[920]: time="2026-11-02T13:16:46.699Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 diskcache=f4ebb59c-6b0c-427f-a76c-ecc760ae1ef2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-02T19:16:46.689" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.689" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969gitlab # [ 397.949021] gitaly[920]: time="2026-11-02T13:16:46.699Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-02T19:16:46.689" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.689" grpc.time_ms=9.757 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1970gitlab # [ 397.962608] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.715Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c1cc99a97704a0ec5bb92fd9","created_at":"2026-11-02T13:16:44.318Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-eaa2354c1a57415a","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"943eb3e09fa57578860a0835","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.321Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c1cc99a97704a0ec5bb92fd9: done: 0.046313 sec","job_status":"done","queue_duration_s":2.348398,"scheduling_latency_s":2.348417,"gitaly_calls":1,"gitaly_duration_s":0.012088,"redis_calls":12,"redis_duration_s":0.0018700000000000001,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000446,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000698,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0278224,"duration_s":0.046313,"completed_at":"2026-11-02T13:16:46.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1971gitlab # [ 398.009629] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:46.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"37dbe5880b4178fff297a93a","created_at":"2026-11-02T13:16:44.432Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-eaa2354c1a57415a","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:85cfb08e32a5734e9f90053871b8027862c691246ef31844d7edd2d6adafc777","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.435Z","job_size_bytes":467,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-37dbe5880b4178fff297a93a: start","job_status":"start","queue_duration_s":2.282455,"scheduling_latency_s":2.282476}1972gitlab # [ 398.027903] gitaly[920]: time="2026-11-02T13:16:46.742Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:16.641" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.741" grpc.time_ms=0.642 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 398.041258] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:46 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 831974gitlab # [ 398.043413] gitaly[920]: time="2026-11-02T13:16:46.749Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FF71TN1QDK3ZNG9GHNMG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:16.648" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.748" grpc.time_ms=0.506 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab # [ 398.057736] gitaly[920]: time="2026-11-02T13:16:46.755Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-02T13:16:56.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:46.745" grpc.time_ms=10.411 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 398.071545] gitaly[920]: time="2026-11-02T13:16:46.793Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-02T13:17:16.692" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.792" grpc.time_ms=0.87 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'1978gitlab # [ 398.084785] gitaly[920]: time="2026-11-02T13:16:46.794Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FF71TN1QDK3ZNG9GHNMG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:56.283" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:46.783" grpc.time_ms=10.708 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979gitlab # [ 398.098225] gitlab-workhorse[826]: time="2026-11-02T13:16:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671980gitlab # [ 398.099962] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 911981gitlab # [ 398.101883] gitaly[920]: time="2026-11-02T13:16:46.802Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=109 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-02T13:17:16.697" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:46.797" grpc.time_ms=4.477 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982gitlab # [ 398.117741] gitaly[920]: time="2026-11-02T13:16:46.843Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:56.089" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:46.829" grpc.time_ms=13.754 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 398.134131] gitaly[920]: time="2026-11-02T13:16:46.966Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FFD2SJ40XKNAJP7PB2W9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:46.954" grpc.time_ms=11.354 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1984gitlab # [ 398.147850] gitaly[920]: time="2026-11-02T13:16:47.059Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFFR9PMTRH135VT2DBDV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.048" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1985gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.07 seconds)1986gitlab # [ 398.158563] gitaly[920]: time="2026-11-02T13:16:47.059Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFFR9PMTRH135VT2DBDV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.048" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1987gitlab # [ 398.172827] nixbot[3407]: {"ts": "2026-11-02T13:16:47.136Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38422 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}1988gitlab # [ 398.175661] gitaly[920]: time="2026-11-02T13:16:47.060Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFFR9PMTRH135VT2DBDV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.048" grpc.time_ms=11.416 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1989gitlab # [ 398.203023] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 861990gitlab # [ 398.205464] gitaly[920]: time="2026-11-02T13:16:47.184Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FFKSFEH4707JXJFCS0P4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.176" grpc.time_ms=7.968 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1991gitlab # [ 398.271809] gitaly[920]: time="2026-11-02T13:16:47.253Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFPKBQYM478V3DH2MCGB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.243" pid=920 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1992gitlab # [ 398.286221] gitaly[920]: time="2026-11-02T13:16:47.253Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFPKBQYM478V3DH2MCGB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.243" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1993gitlab # [ 398.302570] gitlab-workhorse[826]: time="2026-11-02T13:16:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671994gitlab # [ 398.304704] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 661995gitlab # [ 398.306808] gitaly[920]: time="2026-11-02T13:16:47.254Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFPKBQYM478V3DH2MCGB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.243" grpc.time_ms=10.19 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1996gitlab # [ 398.369460] gitaly[920]: time="2026-11-02T13:16:47.351Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FFSVAC5EDGJPCS97WQEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.338" grpc.time_ms=12.39 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc1997gitlab # [ 398.390521] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 601998gitlab # [ 398.436665] gitaly[920]: time="2026-11-02T13:16:47.418Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-02T13:17:17.302" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:47.402" grpc.time_ms=15.593 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999gitlab # [ 398.462777] gitaly[920]: time="2026-11-02T13:16:47.427Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFVS5TWRQBMB9DFQQWFB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.412" pid=920 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=221 span.kind=server system=grpc2000gitlab # [ 398.478856] gitlab-workhorse[826]: time="2026-11-02T13:16:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2212001gitlab # [ 398.481702] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.55.0" 742002gitlab # [ 398.484409] sidekiq-cluster[3287]: {"severity":"WARN","time":"2026-11-02T13:16:47.452Z","class":"CreatePipelineWorker","correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"1012c4552b1ded9281d4a95af15f7a826db59c66","after":"5bcc93bc627dd5c1ed48f56f948c7335c5310516","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"5bcc93bc627dd5c1ed48f56f948c7335c5310516","ref":"refs/heads/master"},"retry":0}2003gitlab # [ 398.493996] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.460Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"37dbe5880b4178fff297a93a","created_at":"2026-11-02T13:16:44.432Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-eaa2354c1a57415a","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:85cfb08e32a5734e9f90053871b8027862c691246ef31844d7edd2d6adafc777","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.435Z","job_size_bytes":467,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-37dbe5880b4178fff297a93a: done: 0.742095 sec","job_status":"done","queue_duration_s":2.282455,"scheduling_latency_s":2.282476,"gitaly_calls":7,"gitaly_duration_s":0.077922,"redis_calls":33,"redis_duration_s":0.003952,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000362,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001689,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.64311,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.742095,"completed_at":"2026-11-02T13:16:47.459Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020439,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2004gitlab # [ 398.546858] gitaly[920]: time="2026-11-02T13:16:47.428Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFVS5TWRQBMB9DFQQWFB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.412" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2005gitlab # [ 398.561446] nixbot[3407]: {"ts": "2026-11-02T13:16:47.497Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "cc58ad1c-fe4d-4d08-9f8d-76880a246582", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "5bcc93bc627dd5c1ed48f56f948c7335c5310516", "pr": null}2006gitlab # [ 398.565295] nixbot[3407]: {"ts": "2026-11-02T13:16:47.499Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38438 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}2007gitlab # [ 398.567383] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.462Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"6e87d82f-394e-4bca-882e-33d200256d99\"}"],"class":"WebHookWorker","jid":"0e3b8211d5d22ce377115289","created_at":"2026-11-02T13:16:44.455Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-eaa2354c1a57415a","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d51779a56af0c57c967b27be3660c7a4a839e3b95a298d31135ffd3afc7d98e0","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.459Z","job_size_bytes":1749,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0e3b8211d5d22ce377115289: start","job_status":"start","queue_duration_s":3.003351,"scheduling_latency_s":3.003388}2008gitlab # [ 398.585365] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.514Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"6e87d82f-394e-4bca-882e-33d200256d99\"}"],"class":"WebHookWorker","jid":"0e3b8211d5d22ce377115289","created_at":"2026-11-02T13:16:44.455Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-eaa2354c1a57415a","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d51779a56af0c57c967b27be3660c7a4a839e3b95a298d31135ffd3afc7d98e0","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.459Z","job_size_bytes":1749,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0e3b8211d5d22ce377115289: done: 0.051848 sec","job_status":"done","queue_duration_s":3.003351,"scheduling_latency_s":3.003388,"redis_calls":14,"redis_duration_s":0.002791,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000337,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000859,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006806360000041423,"cpu_s":0.034568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.051848,"completed_at":"2026-11-02T13:16:47.514Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2009gitlab # [ 398.634215] gitaly[920]: time="2026-11-02T13:16:47.428Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FFVS5TWRQBMB9DFQQWFB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.412" grpc.time_ms=16.116 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2010gitlab # [ 398.649358] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 832011gitlab # [ 398.651500] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.516Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"67c5f4b9ddb8e64cd7a01a33","created_at":"2026-11-02T13:16:44.599Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-6e3191cb47f04043","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.605Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-67c5f4b9ddb8e64cd7a01a33: start","job_status":"start","queue_duration_s":2.910797,"scheduling_latency_s":2.910823}2012gitlab # [ 398.669077] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.580Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"67c5f4b9ddb8e64cd7a01a33","created_at":"2026-11-02T13:16:44.599Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-6e3191cb47f04043","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:44.605Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-67c5f4b9ddb8e64cd7a01a33: done: 0.064444 sec","job_status":"done","queue_duration_s":2.910797,"scheduling_latency_s":2.910823,"redis_calls":19,"redis_duration_s":0.008666,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005147,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000464,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002478,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008915000000000001,"duration_s":0.064444,"completed_at":"2026-11-02T13:16:47.580Z","load_balancing_strategy":"primary","db_duration_s":0.007258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2013gitlab # [ 398.715364] gitaly[920]: time="2026-11-02T13:16:47.437Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6F8MVN4NGCZCF31R9A288 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-02T13:17:17.321" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:47.422" grpc.time_ms=15.115 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2014gitlab # [ 398.729087] gitlab-workhorse[826]: time="2026-11-02T13:16:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672015gitlab # [ 398.731008] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 782016gitlab # [ 398.732999] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.582Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7ea342827de65d55628dd3e6"],"class":"WebHooks::LogExecutionWorker","jid":"b6dc6ca3080b6b9576185fdb","created_at":"2026-11-02T13:16:45.874Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-cd1d81b4ccb04303","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:93700ae8c1d63d1593360716be9cac55fb8efa5a7aab08f31571d99e838cad00","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:45.883Z","job_size_bytes":2593,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b6dc6ca3080b6b9576185fdb: start","job_status":"start","queue_duration_s":1.698869,"scheduling_latency_s":1.698888}2017gitlab # [ 398.750429] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.637Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7ea342827de65d55628dd3e6"],"class":"WebHooks::LogExecutionWorker","jid":"b6dc6ca3080b6b9576185fdb","created_at":"2026-11-02T13:16:45.874Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-cd1d81b4ccb04303","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:93700ae8c1d63d1593360716be9cac55fb8efa5a7aab08f31571d99e838cad00","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:45.883Z","job_size_bytes":2593,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b6dc6ca3080b6b9576185fdb: done: 0.055434 sec","job_status":"done","queue_duration_s":1.698869,"scheduling_latency_s":1.698888,"redis_calls":10,"redis_duration_s":0.00183,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053366,"duration_s":0.055434,"completed_at":"2026-11-02T13:16:47.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2018gitlab # [ 398.796706] gitaly[920]: time="2026-11-02T13:16:47.609Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FG1612MQTTDHR5S4QST7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.596" grpc.time_ms=13.367 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2019gitlab # [ 398.810651] gitaly[920]: time="2026-11-02T13:16:47.687Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG3VBCX4XE8XTNPTDGA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.677" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2020gitlab # [ 398.821828] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.639Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fbd2f38d40a541f7a46d9dcb","created_at":"2026-11-02T13:16:46.445Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-b46e684a6d8b4bd1","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:46.451Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fbd2f38d40a541f7a46d9dcb: start","job_status":"start","queue_duration_s":1.187954,"scheduling_latency_s":1.187978}2021gitlab # [ 398.838295] gitaly[920]: time="2026-11-02T13:16:47.688Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG3VBCX4XE8XTNPTDGA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.677" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2022gitlab # [ 398.852035] gitaly[920]: time="2026-11-02T13:16:47.689Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG3VBCX4XE8XTNPTDGA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:47.677" grpc.time_ms=11.31 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2023gitlab # [ 398.867450] gitaly[920]: time="2026-11-02T13:16:47.763Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:47.762" grpc.time_ms=0.541 pid=920 span.kind=server system=grpc2024gitlab # [ 398.878422] gitaly[920]: time="2026-11-02T13:16:47.792Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:17.691" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:47.791" grpc.time_ms=1.067 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025gitlab # [ 398.891953] gitaly[920]: time="2026-11-02T13:16:47.811Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:47.810" grpc.time_ms=0.588 pid=920 span.kind=server system=grpc2026gitlab # [ 398.902217] gitaly[920]: time="2026-11-02T13:16:47.813Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-02T13:17:17.702" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:47.803" grpc.time_ms=10.591 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027gitlab # [ 398.916499] gitaly[920]: time="2026-11-02T13:16:47.831Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:16:57.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:47.821" grpc.time_ms=10.117 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028gitlab # [ 398.930446] gitaly[920]: time="2026-11-02T13:16:47.835Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:47.835" grpc.time_ms=0.552 pid=920 span.kind=server system=grpc2029gitlab # [ 398.940473] gitaly[920]: time="2026-11-02T13:16:47.843Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 diskcache=246af423-40c7-41b6-aa6e-50944951cfc9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:17.631" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:47.732" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2030gitlab # [ 398.950773] gitaly[920]: time="2026-11-02T13:16:47.843Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=103 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7240 command.oublock=0 command.real_time_ms=107 command.spawn_token_fork_ms=0 command.system_time_ms=53 command.user_time_ms=50 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:17.631" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:47.732" grpc.time_ms=111.216 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2031gitlab # [ 398.964354] gitaly[920]: time="2026-11-02T13:16:47.851Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-02T13:16:57.030" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:47.851" grpc.time_ms=0.255 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032gitlab # [ 398.975357] gitaly[920]: time="2026-11-02T13:16:47.864Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-02T13:16:57.035" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:47.856" grpc.time_ms=8.463 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2033gitlab # [ 398.989237] gitaly[920]: time="2026-11-02T13:16:47.897Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:47.897" grpc.time_ms=0.371 pid=920 span.kind=server system=grpc2034gitlab # [ 398.999075] gitaly[920]: time="2026-11-02T13:16:47.912Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-02T13:17:17.810" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:47.911" grpc.time_ms=1.546 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2035gitlab # [ 399.012306] gitaly[920]: time="2026-11-02T13:16:47.923Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:47.922" grpc.time_ms=0.332 pid=920 span.kind=server system=grpc2036gitlab # [ 399.022019] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.958Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fbd2f38d40a541f7a46d9dcb","created_at":"2026-11-02T13:16:46.445Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-b46e684a6d8b4bd1","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:46.451Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3287,"job_status":"rescheduled","message":"PipelineProcessWorker JID-fbd2f38d40a541f7a46d9dcb: rescheduled"}2037gitlab # [ 399.037473] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.966Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fbd2f38d40a541f7a46d9dcb","created_at":"2026-11-02T13:16:46.445Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-b46e684a6d8b4bd1","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:46.451Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fbd2f38d40a541f7a46d9dcb: done: 0.327358 sec","job_status":"done","queue_duration_s":1.187954,"scheduling_latency_s":1.187978,"gitaly_calls":3,"gitaly_duration_s":0.121203,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.016407,"redis_read_bytes":148,"redis_write_bytes":18011,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000527,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000737,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002662,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.006015,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000276,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000981,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.004682,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.165999,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.2804746,"duration_s":0.327358,"completed_at":"2026-11-02T13:16:47.966Z","load_balancing_strategy":"primary","db_duration_s":0.033209,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2038gitlab # [ 399.091847] gitaly[920]: time="2026-11-02T13:16:47.930Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH diskcache=915db891-6a85-4a98-9b17-d5bd71a2cd1d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-02T13:16:57.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:47.867" pid=920 ref=refs/keep-around/5bcc93bc627dd5c1ed48f56f948c7335c5310516 remote_ip="2001:db8:1::1" revision=5bcc93bc627dd5c1ed48f56f948c7335c5310516 span.kind=server system=grpc username=root2039gitlab # [ 399.103758] nixbot[3407]: {"ts": "2026-11-02T13:16:48.029Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-5bcc93bc627d-6"}2040gitlab # [ 399.106572] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.968Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0e3b8211d5d22ce377115289"],"class":"WebHooks::LogExecutionWorker","jid":"8e6d8ed6a724a34c8a4692f3","created_at":"2026-11-02T13:16:47.501Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-7ca7cd93789a44a5","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:34c0bfa6d94ff44102639788c016ff8c9dfb774281a78f1db513f15ab1d93972","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.505Z","job_size_bytes":2592,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8e6d8ed6a724a34c8a4692f3: start","job_status":"start","queue_duration_s":0.463266,"scheduling_latency_s":0.463292}2041gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516'2042gitlab # [ 399.124144] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.995Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0e3b8211d5d22ce377115289"],"class":"WebHooks::LogExecutionWorker","jid":"8e6d8ed6a724a34c8a4692f3","created_at":"2026-11-02T13:16:47.501Z","trace_propagation_headers":{"sentry-trace":"0f76edb99bd147199b115b06ef43609f-7ca7cd93789a44a5","baggage":"sentry-trace_id=0f76edb99bd147199b115b06ef43609f,sentry-release=Unknown"},"correlation_id":"01M6E6F8MVN4NGCZCF31R9A288","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:34c0bfa6d94ff44102639788c016ff8c9dfb774281a78f1db513f15ab1d93972","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.505Z","job_size_bytes":2592,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8e6d8ed6a724a34c8a4692f3: done: 0.027419 sec","job_status":"done","queue_duration_s":0.463266,"scheduling_latency_s":0.463292,"redis_calls":7,"redis_duration_s":0.001582,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027419,"completed_at":"2026-11-02T13:16:47.995Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043gitlab # [ 399.168827] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 2952044gitlab # [ 399.171271] gitaly[920]: time="2026-11-02T13:16:47.930Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=51 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5118 command.oublock=8 command.real_time_ms=60 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-02T13:16:57.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-02T13:16:47.867" grpc.time_ms=63.389 pid=920 ref=refs/keep-around/5bcc93bc627dd5c1ed48f56f948c7335c5310516 remote_ip="2001:db8:1::1" revision=5bcc93bc627dd5c1ed48f56f948c7335c5310516 span.kind=server system=grpc username=root2045gitlab # [ 399.185917] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:47.997Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"7bf2340e9be52e350cfcfa2e","created_at":"2026-11-02T13:16:47.565Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-dfe6b43805b04ca4","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.571Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-7bf2340e9be52e350cfcfa2e: start","job_status":"start","queue_duration_s":0.425658,"scheduling_latency_s":0.425678}2046gitlab # [ 399.203450] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.096Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"7bf2340e9be52e350cfcfa2e","created_at":"2026-11-02T13:16:47.565Z","trace_propagation_headers":{"sentry-trace":"4b268abdc7534c9cb93d7a607b198fa1-dfe6b43805b04ca4","baggage":"sentry-trace_id=4b268abdc7534c9cb93d7a607b198fa1,sentry-release=Unknown"},"correlation_id":"01M6E6CNGKPXB9YVQ20KNKXK9N","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.571Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-7bf2340e9be52e350cfcfa2e: done: 0.099778 sec","job_status":"done","queue_duration_s":0.425658,"scheduling_latency_s":0.425678,"redis_calls":11,"redis_duration_s":0.002663,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001905,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028426,"duration_s":0.099778,"completed_at":"2026-11-02T13:16:48.096Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047gitlab # [ 399.250077] gitaly[920]: time="2026-11-02T13:16:47.934Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:17.833" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:47.933" grpc.time_ms=0.25 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2048gitlab # [ 399.263233] nixbot[3407]: {"ts": "2026-11-02T13:16:48.189Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-5bcc93bc627d-6", "argv0": "bwrap"}2049gitlab # [ 399.265846] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.101Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"28fe4ed2543a008714e542c5","created_at":"2026-11-02T13:16:47.722Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.724Z","job_size_bytes":23,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-28fe4ed2543a008714e542c5: start","job_status":"start","queue_duration_s":0.376635,"scheduling_latency_s":0.376656}2050gitlab # [ 399.283721] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.159Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"28fe4ed2543a008714e542c5","created_at":"2026-11-02T13:16:47.722Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.724Z","job_size_bytes":23,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-28fe4ed2543a008714e542c5: done: 0.058236 sec","job_status":"done","queue_duration_s":0.376635,"scheduling_latency_s":0.376656,"redis_calls":7,"redis_duration_s":0.005436,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058236,"completed_at":"2026-11-02T13:16:48.159Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015331,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2051gitlab # [ 399.327567] gitaly[920]: time="2026-11-02T13:16:47.948Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FEQJJ1Q0YVZ7HYA3G1R7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:17.847" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:47.948" grpc.time_ms=0.61 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2052gitlab # [ 399.340426] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.161Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dbf8408f4b70ba59855615da","created_at":"2026-11-02T13:16:47.849Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.852Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dbf8408f4b70ba59855615da: start","job_status":"start","queue_duration_s":0.308818,"scheduling_latency_s":0.308843}2053gitlab # [ 399.357390] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.210Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"dbf8408f4b70ba59855615da","created_at":"2026-11-02T13:16:47.849Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.852Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-dbf8408f4b70ba59855615da: done: 0.048945 sec","job_status":"done","queue_duration_s":0.308818,"scheduling_latency_s":0.308843,"redis_calls":7,"redis_duration_s":0.0013570000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048945,"completed_at":"2026-11-02T13:16:48.209Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516', in 0.24 seconds)2055(finished: subtest: push delivers the webhook, in 89.46 seconds)2056subtest: eval and build post commit statuses2057gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2058gitlab # [ 399.401577] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.211Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0dc07c99cecc4755a3728d40","created_at":"2026-11-02T13:16:47.864Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.868Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0dc07c99cecc4755a3728d40: start","job_status":"start","queue_duration_s":0.343133,"scheduling_latency_s":0.343163}2059gitlab # [ 399.418185] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.245Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0dc07c99cecc4755a3728d40","created_at":"2026-11-02T13:16:47.864Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.868Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0dc07c99cecc4755a3728d40: done: 0.033437 sec","job_status":"done","queue_duration_s":0.343133,"scheduling_latency_s":0.343163,"redis_calls":7,"redis_duration_s":0.0013080000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033437,"completed_at":"2026-11-02T13:16:48.245Z","load_balancing_strategy":"primary","db_duration_s":0.006343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2060gitlab # [ 399.461889] nixbot[3407]: {"ts": "2026-11-02T13:16:48.359Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:38444 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1\" 200"}2061gitlab # [ 399.464786] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.246Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"39fcdf40da4c0aa29fc1d566","created_at":"2026-11-02T13:16:47.878Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.880Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-39fcdf40da4c0aa29fc1d566: start","job_status":"start","queue_duration_s":0.366142,"scheduling_latency_s":0.366165}2062gitlab # [ 399.481947] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.281Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"39fcdf40da4c0aa29fc1d566","created_at":"2026-11-02T13:16:47.878Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.880Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-39fcdf40da4c0aa29fc1d566: done: 0.034685 sec","job_status":"done","queue_duration_s":0.366142,"scheduling_latency_s":0.366165,"redis_calls":10,"redis_duration_s":0.0020670000000000003,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00101,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000494,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034685,"completed_at":"2026-11-02T13:16:48.281Z","load_balancing_strategy":"primary","db_duration_s":0.00652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 399.527400] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.282Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"792e9a917a6dded77d67ebd3","created_at":"2026-11-02T13:16:47.881Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.883Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-792e9a917a6dded77d67ebd3: start","job_status":"start","queue_duration_s":0.399439,"scheduling_latency_s":0.39946}2064gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.15 seconds)2065gitlab # [ 399.544441] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.311Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"792e9a917a6dded77d67ebd3","created_at":"2026-11-02T13:16:47.881Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.883Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-792e9a917a6dded77d67ebd3: done: 0.028636 sec","job_status":"done","queue_duration_s":0.399439,"scheduling_latency_s":0.39946,"redis_calls":7,"redis_duration_s":0.001372,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000346,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028636,"completed_at":"2026-11-02T13:16:48.311Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2066gitlab # [ 399.588450] gitaly[920]: time="2026-11-02T13:16:48.490Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FGWY1JWWHSFF3TXSCN7E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:18.088" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:48.489" grpc.time_ms=1.547 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 399.601871] nix-daemon[3565]: accepted connection from pid 4655, user nixbot2068gitlab # [ 399.602880] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.313Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e541de31aca4f218aaaf88ab","created_at":"2026-11-02T13:16:47.884Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.886Z","job_size_bytes":19,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e541de31aca4f218aaaf88ab: start","job_status":"start","queue_duration_s":0.426511,"scheduling_latency_s":0.426533}2069gitlab # [ 399.619706] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.339Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e541de31aca4f218aaaf88ab","created_at":"2026-11-02T13:16:47.884Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.886Z","job_size_bytes":19,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e541de31aca4f218aaaf88ab: done: 0.027028 sec","job_status":"done","queue_duration_s":0.426511,"scheduling_latency_s":0.426533,"redis_calls":8,"redis_duration_s":0.001577,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027028,"completed_at":"2026-11-02T13:16:48.339Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 399.663343] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1122071gitlab # [ 399.666233] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.341Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f9e5a154013a90a26b0256a3","created_at":"2026-11-02T13:16:47.887Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.889Z","job_size_bytes":66,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f9e5a154013a90a26b0256a3: start","job_status":"start","queue_duration_s":0.451721,"scheduling_latency_s":0.451742}2072gitlab # [ 399.684177] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.383Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f9e5a154013a90a26b0256a3","created_at":"2026-11-02T13:16:47.887Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.889Z","job_size_bytes":66,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f9e5a154013a90a26b0256a3: done: 0.041586 sec","job_status":"done","queue_duration_s":0.451721,"scheduling_latency_s":0.451742,"redis_calls":7,"redis_duration_s":0.008307,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001428,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006879,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041586,"completed_at":"2026-11-02T13:16:48.382Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2073gitlab # [ 399.730132] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.385Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4825f2cd4ac77b2ccbe4b95b","created_at":"2026-11-02T13:16:47.890Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.893Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4825f2cd4ac77b2ccbe4b95b: start","job_status":"start","queue_duration_s":0.491357,"scheduling_latency_s":0.491412}2074gitlab # [ 399.747087] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.429Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4825f2cd4ac77b2ccbe4b95b","created_at":"2026-11-02T13:16:47.890Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.893Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4825f2cd4ac77b2ccbe4b95b: done: 0.044242 sec","job_status":"done","queue_duration_s":0.491357,"scheduling_latency_s":0.491412,"redis_calls":7,"redis_duration_s":0.001572,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044242,"completed_at":"2026-11-02T13:16:48.429Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075gitlab # [ 399.790998] gitaly[920]: time="2026-11-02T13:16:48.700Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-02T13:17:18.598" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:48.699" grpc.time_ms=1.146 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2076gitlab # [ 399.804206] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.431Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"362b6f8b08ce4ae6e92e789c","created_at":"2026-11-02T13:16:47.894Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.896Z","job_size_bytes":6,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-362b6f8b08ce4ae6e92e789c: start","job_status":"start","queue_duration_s":0.534635,"scheduling_latency_s":0.534663}2077gitlab # [ 399.820492] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.453Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"362b6f8b08ce4ae6e92e789c","created_at":"2026-11-02T13:16:47.894Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.896Z","job_size_bytes":6,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-362b6f8b08ce4ae6e92e789c: done: 0.022071 sec","job_status":"done","queue_duration_s":0.534635,"scheduling_latency_s":0.534663,"redis_calls":7,"redis_duration_s":0.001533,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022071,"completed_at":"2026-11-02T13:16:48.453Z","load_balancing_strategy":"primary","db_duration_s":0.000483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078gitlab # [ 399.863664] gitaly[920]: time="2026-11-02T13:16:48.755Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FG7EQBZ1TEPMM8PTFVSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:18.653" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:48.754" grpc.time_ms=0.979 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2079gitlab # [ 399.877181] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.455Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a73868b612b7d560edcdb690","created_at":"2026-11-02T13:16:47.897Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.899Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a73868b612b7d560edcdb690: start","job_status":"start","queue_duration_s":0.555939,"scheduling_latency_s":0.555968}2080gitlab # [ 399.893029] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a73868b612b7d560edcdb690","created_at":"2026-11-02T13:16:47.897Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.899Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a73868b612b7d560edcdb690: done: 0.034839 sec","job_status":"done","queue_duration_s":0.555939,"scheduling_latency_s":0.555968,"redis_calls":7,"redis_duration_s":0.001663,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034839,"completed_at":"2026-11-02T13:16:48.489Z","load_balancing_strategy":"primary","db_duration_s":0.004017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 399.936755] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.491Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8060ec6d5c6cf65b13db48b2","created_at":"2026-11-02T13:16:47.935Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-59f3191987544ea1","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.939Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8060ec6d5c6cf65b13db48b2: start","job_status":"start","queue_duration_s":0.551798,"scheduling_latency_s":0.551827}2082gitlab # [ 399.953999] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.525Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8060ec6d5c6cf65b13db48b2","created_at":"2026-11-02T13:16:47.935Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-59f3191987544ea1","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.939Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8060ec6d5c6cf65b13db48b2: done: 0.034235 sec","job_status":"done","queue_duration_s":0.551798,"scheduling_latency_s":0.551827,"redis_calls":7,"redis_duration_s":0.0010370000000000002,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034235,"completed_at":"2026-11-02T13:16:48.525Z","load_balancing_strategy":"primary","db_duration_s":0.004403,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2083gitlab # [ 399.998218] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.526Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"88f2b29028da49a1517cf16e","created_at":"2026-11-02T13:16:47.940Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-59f3191987544ea1","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.943Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-88f2b29028da49a1517cf16e: start","job_status":"start","queue_duration_s":0.582981,"scheduling_latency_s":0.583}2084gitlab # [ 400.016521] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.583Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"88f2b29028da49a1517cf16e","created_at":"2026-11-02T13:16:47.940Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-59f3191987544ea1","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.943Z","job_size_bytes":65,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-88f2b29028da49a1517cf16e: done: 0.057115 sec","job_status":"done","queue_duration_s":0.582981,"scheduling_latency_s":0.583,"redis_calls":10,"redis_duration_s":0.002342,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057115,"completed_at":"2026-11-02T13:16:48.583Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 400.061707] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"510b5a10810ad417cb2617cf","created_at":"2026-11-02T13:16:47.959Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.961Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-510b5a10810ad417cb2617cf: start","job_status":"start","queue_duration_s":0.62418,"scheduling_latency_s":0.6242}2086gitlab # [ 400.078020] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.613Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"510b5a10810ad417cb2617cf","created_at":"2026-11-02T13:16:47.959Z","trace_propagation_headers":{"sentry-trace":"bd11db230853439292890ad120087745-d1c7e2aa62724bc4","baggage":"sentry-trace_id=bd11db230853439292890ad120087745,sentry-release=Unknown"},"correlation_id":"01M6E6FEQJJ1Q0YVZ7HYA3G1R7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:47.961Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-510b5a10810ad417cb2617cf: done: 0.028079 sec","job_status":"done","queue_duration_s":0.62418,"scheduling_latency_s":0.6242,"redis_calls":8,"redis_duration_s":0.00163,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028079,"completed_at":"2026-11-02T13:16:48.613Z","load_balancing_strategy":"primary","db_duration_s":0.004098,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2087gitlab # [ 400.122622] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.615Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ba852d37d3e9a4293aad2e07","created_at":"2026-11-02T13:16:47.997Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-59f3191987544ea1","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:48.000Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ba852d37d3e9a4293aad2e07: start","job_status":"start","queue_duration_s":0.614473,"scheduling_latency_s":0.614493}2088gitlab # [ 400.139285] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.773Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ba852d37d3e9a4293aad2e07","created_at":"2026-11-02T13:16:47.997Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-59f3191987544ea1","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:48.000Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ba852d37d3e9a4293aad2e07: done: 0.15797 sec","job_status":"done","queue_duration_s":0.614473,"scheduling_latency_s":0.614493,"gitaly_calls":2,"gitaly_duration_s":0.008489,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008046,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001264,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001333,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000254,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00188,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000479,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000545,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120511,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12081426,"duration_s":0.15797,"completed_at":"2026-11-02T13:16:48.772Z","load_balancing_strategy":"primary","db_duration_s":0.030669,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2089gitlab # [ 400.192733] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.775Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7919e1686a260789a9207f42","created_at":"2026-11-02T13:16:48.680Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-47595bece683435a","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:48.683Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7919e1686a260789a9207f42: start","job_status":"start","queue_duration_s":0.092075,"scheduling_latency_s":0.092105}2090gitlab # [ 400.209643] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:48.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7919e1686a260789a9207f42","created_at":"2026-11-02T13:16:48.680Z","trace_propagation_headers":{"sentry-trace":"f1158da895ac4751af22e897861639d5-47595bece683435a","baggage":"sentry-trace_id=f1158da895ac4751af22e897861639d5,sentry-release=Unknown"},"correlation_id":"01M6E6FG7EQBZ1TEPMM8PTFVSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:48.683Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7919e1686a260789a9207f42: done: 0.051632 sec","job_status":"done","queue_duration_s":0.092075,"scheduling_latency_s":0.092105,"redis_calls":7,"redis_duration_s":0.001767,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051632,"completed_at":"2026-11-02T13:16:48.826Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2092gitlab # [ 400.667449] gitaly[920]: time="2026-11-02T13:16:49.648Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FJ1BQCWBFX0HAGY6HJWS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:19.247" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:49.647" grpc.time_ms=0.849 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.15 seconds)2094gitlab # [ 400.704375] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1052095gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2096gitlab # [ 401.830646] gitaly[920]: time="2026-11-02T13:16:50.811Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FK515ZNV17NA5JBF281H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:20.410" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:50.810" grpc.time_ms=0.896 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.18 seconds)2098gitlab # [ 401.879489] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1382099gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2100gitlab # [ 402.998429] gitaly[920]: time="2026-11-02T13:16:51.979Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FM9SWEGBHQMA9H4JFM1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:21.577" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:51.978" grpc.time_ms=0.854 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2101gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.16 seconds)2102gitlab # [ 403.037219] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1202103gitlab # [ 403.661045] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:52.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d89fb84456276fac7343a4ea","created_at":"2026-11-02T13:16:52.589Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3549f846eced729a0f483d91fce1a80b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.638Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d89fb84456276fac7343a4ea: start","job_status":"start","queue_duration_s":0.00293,"scheduling_latency_s":0.002985}2104gitlab # [ 403.749887] nix-daemon[3565]: accepted connection from pid 4692, user nixbot2105gitlab # [ 403.773106] nix-daemon[3565]: accepted connection from pid 4655, user nixbot2106gitlab # [ 403.826591] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:52.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d89fb84456276fac7343a4ea","created_at":"2026-11-02T13:16:52.589Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3549f846eced729a0f483d91fce1a80b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.638Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d89fb84456276fac7343a4ea: done: 0.166022 sec","job_status":"done","queue_duration_s":0.00293,"scheduling_latency_s":0.002985,"redis_calls":11,"redis_duration_s":0.033436999999999995,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.019814,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010258,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003365,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.166022,"completed_at":"2026-11-02T13:16:52.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2107gitlab # [ 403.887666] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:52.818Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"4954cf6c531f15917ca979e5","created_at":"2026-11-02T13:16:52.669Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e6d5f396c36a3181bbb3115bf2002b8c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.690Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-4954cf6c531f15917ca979e5: start","job_status":"start","queue_duration_s":0.128421,"scheduling_latency_s":0.128471}2108gitlab # [ 403.935671] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:52.916Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"4954cf6c531f15917ca979e5","created_at":"2026-11-02T13:16:52.669Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e6d5f396c36a3181bbb3115bf2002b8c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.690Z","job_size_bytes":3,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-4954cf6c531f15917ca979e5: done: 0.098238 sec","job_status":"done","queue_duration_s":0.128421,"scheduling_latency_s":0.128471,"redis_calls":10,"redis_duration_s":0.029464,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.012076,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0031028,"duration_s":0.098238,"completed_at":"2026-11-02T13:16:52.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109gitlab # [ 403.997862] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:52.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"41182c1db0110bedb0c96649","created_at":"2026-11-02T13:16:52.698Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d90c70c4637a4613000c416d777cad4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.710Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-41182c1db0110bedb0c96649: start","job_status":"start","queue_duration_s":0.25548,"scheduling_latency_s":0.255525}2110gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2111gitlab # [ 404.158818] gitaly[920]: time="2026-11-02T13:16:53.140Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FNE2PH7MRFCV8NXKYJHK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:23.038" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:53.139" grpc.time_ms=0.763 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.18 seconds)2113gitlab # [ 404.217426] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1392114gitlab # [ 405.133410] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.113Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"41182c1db0110bedb0c96649","created_at":"2026-11-02T13:16:52.698Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d90c70c4637a4613000c416d777cad4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.710Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-41182c1db0110bedb0c96649: done: 1.147258 sec","job_status":"done","queue_duration_s":0.25548,"scheduling_latency_s":0.255525,"redis_calls":13,"redis_duration_s":0.014986,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005857,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00343,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005699,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.189,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.189,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.705555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00291968,"exclusive_lock_hold_duration_s":1.03367936,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":1.147258,"completed_at":"2026-11-02T13:16:54.113Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.438147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115gitlab # [ 405.135632] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"fbee728ac1151746c1e5c85b","created_at":"2026-11-02T13:16:52.729Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9fa67b6ca68bcff5c0c266ec748813f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.753Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-fbee728ac1151746c1e5c85b: start","job_status":"start","queue_duration_s":1.363157,"scheduling_latency_s":1.3632}2116gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2117gitlab # [ 405.221280] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"fbee728ac1151746c1e5c85b","created_at":"2026-11-02T13:16:52.729Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9fa67b6ca68bcff5c0c266ec748813f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.753Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-fbee728ac1151746c1e5c85b: done: 0.049716 sec","job_status":"done","queue_duration_s":1.363157,"scheduling_latency_s":1.3632,"redis_calls":9,"redis_duration_s":0.003966,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000552,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000602,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049716,"completed_at":"2026-11-02T13:16:54.166Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118gitlab # [ 405.270451] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"f7407333fb401c66e1c8179f","created_at":"2026-11-02T13:16:52.781Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.801Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-f7407333fb401c66e1c8179f: start","job_status":"start","queue_duration_s":1.366966,"scheduling_latency_s":1.366994}2119gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.16 seconds)2120gitlab # [ 405.320675] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"f7407333fb401c66e1c8179f","created_at":"2026-11-02T13:16:52.781Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.801Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-f7407333fb401c66e1c8179f: done: 0.133512 sec","job_status":"done","queue_duration_s":1.366966,"scheduling_latency_s":1.366994,"redis_calls":65,"redis_duration_s":0.015653999999999998,"redis_read_bytes":114,"redis_write_bytes":41282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000852,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.002368,"redis_queues_read_bytes":3,"redis_queues_write_bytes":36985,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.011805,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000629,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.117188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003519,"duration_s":0.133512,"completed_at":"2026-11-02T13:16:54.301Z","load_balancing_strategy":"primary","db_duration_s":0.001479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 405.378644] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7e74cf70bcc85fbd095b96c9","created_at":"2026-11-02T13:16:52.812Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f527df6da633616fb32f804dbdca454","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-02T13:16:52.826Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7e74cf70bcc85fbd095b96c9: start","job_status":"start","queue_duration_s":1.477101,"scheduling_latency_s":1.477131}2122gitlab # [ 405.393461] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.318Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2123gitlab # [ 405.397613] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.330Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7e74cf70bcc85fbd095b96c9","created_at":"2026-11-02T13:16:52.812Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f527df6da633616fb32f804dbdca454","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-02T13:16:52.826Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7e74cf70bcc85fbd095b96c9: done: 0.026903 sec","job_status":"done","queue_duration_s":1.477101,"scheduling_latency_s":1.477131,"redis_calls":9,"redis_duration_s":0.002006,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026903,"completed_at":"2026-11-02T13:16:54.330Z","load_balancing_strategy":"primary","db_duration_s":0.001885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 405.442228] gitaly[920]: time="2026-11-02T13:16:54.307Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FPJTZPSCE1559Y7X8F1P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:24.006" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:54.306" grpc.time_ms=0.952 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2125gitlab # [ 405.455743] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1162126gitlab # [ 405.458704] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"251b66185ef33ff75c346a2d","created_at":"2026-11-02T13:16:52.833Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e276fbee42d789f677e08df2b2cec29","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.855Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-251b66185ef33ff75c346a2d: start","job_status":"start","queue_duration_s":1.476374,"scheduling_latency_s":1.4764}2127gitlab # [ 405.473567] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"251b66185ef33ff75c346a2d","created_at":"2026-11-02T13:16:52.833Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e276fbee42d789f677e08df2b2cec29","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.855Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-251b66185ef33ff75c346a2d: done: 0.04601 sec","job_status":"done","queue_duration_s":1.476374,"scheduling_latency_s":1.4764,"redis_calls":12,"redis_duration_s":0.002906,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001262,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01339176,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.04601,"completed_at":"2026-11-02T13:16:54.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005292,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2128gitlab # [ 405.519383] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e8220d87f7469b9e0c38a23d","created_at":"2026-11-02T13:16:52.885Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30d06ef6b08b96bfc070ef31814e3c91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-02T13:16:52.910Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e8220d87f7469b9e0c38a23d: start","job_status":"start","queue_duration_s":1.469572,"scheduling_latency_s":1.469597}2129gitlab # [ 405.533531] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.394Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2130gitlab # [ 405.536918] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e8220d87f7469b9e0c38a23d","created_at":"2026-11-02T13:16:52.885Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30d06ef6b08b96bfc070ef31814e3c91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-02T13:16:52.910Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e8220d87f7469b9e0c38a23d: done: 0.0248 sec","job_status":"done","queue_duration_s":1.469572,"scheduling_latency_s":1.469597,"redis_calls":9,"redis_duration_s":0.0019130000000000002,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0248,"completed_at":"2026-11-02T13:16:54.404Z","load_balancing_strategy":"primary","db_duration_s":0.000613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131gitlab # [ 405.579167] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"5c6274167498651c2b1a9ba5","created_at":"2026-11-02T13:16:52.961Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a74c28697b55ec885ff15b54cfa3d98d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.973Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-5c6274167498651c2b1a9ba5: start","job_status":"start","queue_duration_s":1.432987,"scheduling_latency_s":1.433013}2132gitlab # [ 405.594249] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"5c6274167498651c2b1a9ba5","created_at":"2026-11-02T13:16:52.961Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a74c28697b55ec885ff15b54cfa3d98d","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.973Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-5c6274167498651c2b1a9ba5: done: 0.036253 sec","job_status":"done","queue_duration_s":1.432987,"scheduling_latency_s":1.433013,"redis_calls":9,"redis_duration_s":0.001876,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000359,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036253,"completed_at":"2026-11-02T13:16:54.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 405.638507] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d1c7e8738e97154349fbf35f","created_at":"2026-11-02T13:16:52.979Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"847b2d6c0fe709820b09305d30c6383b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.999Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d1c7e8738e97154349fbf35f: start","job_status":"start","queue_duration_s":1.44462,"scheduling_latency_s":1.444646}2134gitlab # [ 405.653373] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"d1c7e8738e97154349fbf35f","created_at":"2026-11-02T13:16:52.979Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"847b2d6c0fe709820b09305d30c6383b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:52.999Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-d1c7e8738e97154349fbf35f: done: 0.030618 sec","job_status":"done","queue_duration_s":1.44462,"scheduling_latency_s":1.444646,"redis_calls":11,"redis_duration_s":0.0022340000000000003,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000805,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006106,"exclusive_lock_hold_duration_s":0.00644838,"duration_s":0.030618,"completed_at":"2026-11-02T13:16:54.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135gitlab # [ 405.696974] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7c41ce00ec3b334c2bac5faf","created_at":"2026-11-02T13:16:53.042Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f10ab5667fc18341039d4bbe42c0e794","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.066Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7c41ce00ec3b334c2bac5faf: start","job_status":"start","queue_duration_s":1.410107,"scheduling_latency_s":1.410131}2136gitlab # [ 405.711733] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"7c41ce00ec3b334c2bac5faf","created_at":"2026-11-02T13:16:53.042Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f10ab5667fc18341039d4bbe42c0e794","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.066Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-7c41ce00ec3b334c2bac5faf: done: 0.028354 sec","job_status":"done","queue_duration_s":1.410107,"scheduling_latency_s":1.410131,"redis_calls":11,"redis_duration_s":0.003209,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001896,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006242,"exclusive_lock_hold_duration_s":0.00372916,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.028354,"completed_at":"2026-11-02T13:16:54.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2137gitlab # [ 405.756492] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0ff4945f95d018e0f33b55fc","created_at":"2026-11-02T13:16:53.095Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90106a7f10b9b48f139a43575b939686","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.115Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0ff4945f95d018e0f33b55fc: start","job_status":"start","queue_duration_s":1.392231,"scheduling_latency_s":1.392255}2138gitlab # [ 405.771451] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0ff4945f95d018e0f33b55fc","created_at":"2026-11-02T13:16:53.095Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90106a7f10b9b48f139a43575b939686","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.115Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0ff4945f95d018e0f33b55fc: done: 0.052934 sec","job_status":"done","queue_duration_s":1.392231,"scheduling_latency_s":1.392255,"redis_calls":11,"redis_duration_s":0.002435,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008019,"exclusive_lock_hold_duration_s":0.01622772,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.052934,"completed_at":"2026-11-02T13:16:54.560Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139gitlab # [ 405.817575] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b6b7e241752bc8f49fa281c1","created_at":"2026-11-02T13:16:53.122Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.156Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b6b7e241752bc8f49fa281c1: start","job_status":"start","queue_duration_s":1.405502,"scheduling_latency_s":1.405527}2140gitlab # [ 405.832333] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b6b7e241752bc8f49fa281c1","created_at":"2026-11-02T13:16:53.122Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.156Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b6b7e241752bc8f49fa281c1: done: 0.042824 sec","job_status":"done","queue_duration_s":1.405502,"scheduling_latency_s":1.405527,"redis_calls":18,"redis_duration_s":0.003396,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000741,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000292,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001775,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042824,"completed_at":"2026-11-02T13:16:54.604Z","load_balancing_strategy":"primary","db_duration_s":0.00222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 405.875875] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e1d0f48cb6a191e5319356e5","created_at":"2026-11-02T13:16:53.187Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7afd36feed8d8623b3fa159097acd6df","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.206Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e1d0f48cb6a191e5319356e5: start","job_status":"start","queue_duration_s":1.399687,"scheduling_latency_s":1.399709}2142gitlab # [ 405.890986] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e1d0f48cb6a191e5319356e5","created_at":"2026-11-02T13:16:53.187Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7afd36feed8d8623b3fa159097acd6df","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.206Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e1d0f48cb6a191e5319356e5: done: 0.055616 sec","job_status":"done","queue_duration_s":1.399687,"scheduling_latency_s":1.399709,"redis_calls":13,"redis_duration_s":0.003301,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000803,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001366,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00075004,"exclusive_lock_hold_duration_s":0.027476,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.055616,"completed_at":"2026-11-02T13:16:54.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 405.937156] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.663Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d1f2cc157593a705e04fd811","created_at":"2026-11-02T13:16:53.239Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"875375854675f321e597f10f0dc2caee","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.264Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d1f2cc157593a705e04fd811: start","job_status":"start","queue_duration_s":1.39921,"scheduling_latency_s":1.399233}2144gitlab # [ 405.951336] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d1f2cc157593a705e04fd811","created_at":"2026-11-02T13:16:53.239Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"875375854675f321e597f10f0dc2caee","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.264Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d1f2cc157593a705e04fd811: done: 0.039987 sec","job_status":"done","queue_duration_s":1.39921,"scheduling_latency_s":1.399233,"redis_calls":9,"redis_duration_s":0.002225,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.039987,"completed_at":"2026-11-02T13:16:54.703Z","load_balancing_strategy":"primary","db_duration_s":0.012159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 405.995040] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"56c01741153c68dd2bea5e49","created_at":"2026-11-02T13:16:53.292Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b9e51e168193b138f5f4ef5dfe2d035","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.309Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-56c01741153c68dd2bea5e49: start","job_status":"start","queue_duration_s":1.395782,"scheduling_latency_s":1.395804}2146gitlab # [ 406.010190] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.719Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2147gitlab # [ 406.013694] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"56c01741153c68dd2bea5e49","created_at":"2026-11-02T13:16:53.292Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b9e51e168193b138f5f4ef5dfe2d035","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.309Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-56c01741153c68dd2bea5e49: done: 0.021991 sec","job_status":"done","queue_duration_s":1.395782,"scheduling_latency_s":1.395804,"redis_calls":9,"redis_duration_s":0.001711,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021991,"completed_at":"2026-11-02T13:16:54.727Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000946,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 406.058605] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"35b2dd81ea5239c41f2a922d","created_at":"2026-11-02T13:16:53.547Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7647a28f931f969f51e075f100735d0c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.566Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-35b2dd81ea5239c41f2a922d: start","job_status":"start","queue_duration_s":1.162618,"scheduling_latency_s":1.162639}2149gitlab # [ 406.073639] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.751Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"35b2dd81ea5239c41f2a922d","created_at":"2026-11-02T13:16:53.547Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7647a28f931f969f51e075f100735d0c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.566Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-35b2dd81ea5239c41f2a922d: done: 0.022354 sec","job_status":"done","queue_duration_s":1.162618,"scheduling_latency_s":1.162639,"redis_calls":9,"redis_duration_s":0.00175,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022354,"completed_at":"2026-11-02T13:16:54.751Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2150gitlab # [ 406.117103] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:54.752Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"716c8404e6e683f5fc58f735","created_at":"2026-11-02T13:16:53.586Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1397cb0832072d6d0d905986af56df2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.609Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-716c8404e6e683f5fc58f735: start","job_status":"start","queue_duration_s":1.143306,"scheduling_latency_s":1.143328}2151gitlab # [ 406.131580] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.106Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"716c8404e6e683f5fc58f735","created_at":"2026-11-02T13:16:53.586Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1397cb0832072d6d0d905986af56df2","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.609Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-716c8404e6e683f5fc58f735: done: 0.35405 sec","job_status":"done","queue_duration_s":1.143306,"scheduling_latency_s":1.143328,"redis_calls":564,"redis_duration_s":0.076086,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.074669,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.314355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.35405,"completed_at":"2026-11-02T13:16:55.106Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152gitlab # [ 406.175102] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.108Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"31d9817b8d71b2f677101073","created_at":"2026-11-02T13:16:53.637Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e39c6c21452d46cd20caacef94706f78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.653Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-31d9817b8d71b2f677101073: start","job_status":"start","queue_duration_s":1.454339,"scheduling_latency_s":1.454367}2153gitlab # [ 406.190387] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.123Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2154gitlab # [ 406.194363] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.132Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"31d9817b8d71b2f677101073","created_at":"2026-11-02T13:16:53.637Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e39c6c21452d46cd20caacef94706f78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.653Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-31d9817b8d71b2f677101073: done: 0.02428 sec","job_status":"done","queue_duration_s":1.454339,"scheduling_latency_s":1.454367,"redis_calls":9,"redis_duration_s":0.0018670000000000002,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000384,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02428,"completed_at":"2026-11-02T13:16:55.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000724,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2155gitlab # [ 406.239089] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9b4c54c1b6a5ac5c20fa78d7","created_at":"2026-11-02T13:16:53.700Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c06d5f74cfc559c74332ffd5a0eeffa8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.721Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9b4c54c1b6a5ac5c20fa78d7: start","job_status":"start","queue_duration_s":1.412864,"scheduling_latency_s":1.412891}2156gitlab # [ 406.254014] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.148Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2157gitlab # [ 406.257350] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9b4c54c1b6a5ac5c20fa78d7","created_at":"2026-11-02T13:16:53.700Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c06d5f74cfc559c74332ffd5a0eeffa8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.721Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9b4c54c1b6a5ac5c20fa78d7: done: 0.024081 sec","job_status":"done","queue_duration_s":1.412864,"scheduling_latency_s":1.412891,"redis_calls":9,"redis_duration_s":0.001901,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000438,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024081,"completed_at":"2026-11-02T13:16:55.158Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158gitlab # [ 406.300584] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.160Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"071fb84b06ba580a601bdd2e","created_at":"2026-11-02T13:16:53.740Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9fc369bba48cbca1fb02c746fc636a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.758Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-071fb84b06ba580a601bdd2e: start","job_status":"start","queue_duration_s":1.401469,"scheduling_latency_s":1.401498}2159gitlab # [ 406.315346] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.174Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2160gitlab # [ 406.319183] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"071fb84b06ba580a601bdd2e","created_at":"2026-11-02T13:16:53.740Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9fc369bba48cbca1fb02c746fc636a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.758Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-071fb84b06ba580a601bdd2e: done: 0.023593 sec","job_status":"done","queue_duration_s":1.401469,"scheduling_latency_s":1.401498,"redis_calls":9,"redis_duration_s":0.001936,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023593,"completed_at":"2026-11-02T13:16:55.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2161gitlab # [ 406.362745] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1d26b3162b7dfc7166865a0e","created_at":"2026-11-02T13:16:53.784Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c357d5740b1d97dd2d2ebb8168a2563e","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.805Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1d26b3162b7dfc7166865a0e: start","job_status":"start","queue_duration_s":1.380176,"scheduling_latency_s":1.380202}2162gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2163gitlab # [ 406.376456] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.209Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1d26b3162b7dfc7166865a0e","created_at":"2026-11-02T13:16:53.784Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c357d5740b1d97dd2d2ebb8168a2563e","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.805Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1d26b3162b7dfc7166865a0e: done: 0.024157 sec","job_status":"done","queue_duration_s":1.380176,"scheduling_latency_s":1.380202,"redis_calls":11,"redis_duration_s":0.002194,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000405,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024157,"completed_at":"2026-11-02T13:16:55.209Z","load_balancing_strategy":"primary","db_duration_s":0.000587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 406.418783] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"524a44e56cc8b6d61ef2e5b8","created_at":"2026-11-02T13:16:53.831Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e703526cab7798928ebde4f65d4adb36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-02T13:16:53.854Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-524a44e56cc8b6d61ef2e5b8: start","job_status":"start","queue_duration_s":1.356628,"scheduling_latency_s":1.35665}2165gitlab # [ 406.432355] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"524a44e56cc8b6d61ef2e5b8","created_at":"2026-11-02T13:16:53.831Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e703526cab7798928ebde4f65d4adb36","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-02T13:16:53.854Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-524a44e56cc8b6d61ef2e5b8: done: 0.043676 sec","job_status":"done","queue_duration_s":1.356628,"scheduling_latency_s":1.35665,"redis_calls":9,"redis_duration_s":0.002027,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043676,"completed_at":"2026-11-02T13:16:55.254Z","load_balancing_strategy":"primary","db_duration_s":0.005345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166gitlab # [ 406.475062] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.256Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b96e6c82db7c3fe0e7e34dfb","created_at":"2026-11-02T13:16:53.883Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"356086cf1227965f0cc170307caee439","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.906Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b96e6c82db7c3fe0e7e34dfb: start","job_status":"start","queue_duration_s":1.350255,"scheduling_latency_s":1.350281}2167gitlab # [ 406.489697] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.294Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"b96e6c82db7c3fe0e7e34dfb","created_at":"2026-11-02T13:16:53.883Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"356086cf1227965f0cc170307caee439","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.906Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-b96e6c82db7c3fe0e7e34dfb: done: 0.037584 sec","job_status":"done","queue_duration_s":1.350255,"scheduling_latency_s":1.350281,"redis_calls":9,"redis_duration_s":0.001887,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000381,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037584,"completed_at":"2026-11-02T13:16:55.294Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004717,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 406.532750] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.296Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"97a5cbd5d499e4beb50de357","created_at":"2026-11-02T13:16:53.921Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.943Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-97a5cbd5d499e4beb50de357: start","job_status":"start","queue_duration_s":1.352621,"scheduling_latency_s":1.352648}2169gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.18 seconds)2170gitlab # [ 406.546636] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"97a5cbd5d499e4beb50de357","created_at":"2026-11-02T13:16:53.921Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.943Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-97a5cbd5d499e4beb50de357: done: 0.170609 sec","job_status":"done","queue_duration_s":1.352621,"scheduling_latency_s":1.352648,"redis_calls":70,"redis_duration_s":0.022987,"redis_read_bytes":119,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002925,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000989,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.01842,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000653,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.147113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.170609,"completed_at":"2026-11-02T13:16:55.466Z","load_balancing_strategy":"primary","db_duration_s":0.002289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2171gitlab # [ 406.590186] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.470Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"32e5f4323ecb603abbb56d96","created_at":"2026-11-02T13:16:53.968Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2f62825e806d88ecb16c40e31969f91","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.990Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-32e5f4323ecb603abbb56d96: start","job_status":"start","queue_duration_s":1.479666,"scheduling_latency_s":1.479705}2172gitlab # [ 406.604620] gitaly[920]: time="2026-11-02T13:16:55.481Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FQQE5CY7NKJ5T77CXQPG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:25.079" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:55.480" grpc.time_ms=0.882 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2173gitlab # [ 406.618031] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1222174gitlab # [ 406.620791] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"32e5f4323ecb603abbb56d96","created_at":"2026-11-02T13:16:53.968Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-6eea50acad6a4762","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2f62825e806d88ecb16c40e31969f91","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-02T13:16:53.990Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-32e5f4323ecb603abbb56d96: done: 0.050148 sec","job_status":"done","queue_duration_s":1.479666,"scheduling_latency_s":1.479705,"redis_calls":11,"redis_duration_s":0.005587,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003153,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00082636,"exclusive_lock_hold_duration_s":0.01134806,"duration_s":0.050148,"completed_at":"2026-11-02T13:16:55.519Z","load_balancing_strategy":"primary","db_duration_s":0.005636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2175gitlab # [ 406.664362] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07de87dfcf5b7f1329c95548","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07de87dfcf5b7f1329c95548: start","job_status":"start","queue_duration_s":1.232374,"scheduling_latency_s":1.232403}2176gitlab # [ 406.676924] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"07de87dfcf5b7f1329c95548","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-07de87dfcf5b7f1329c95548: done: 0.048961 sec","job_status":"done","queue_duration_s":1.232374,"scheduling_latency_s":1.232403,"redis_calls":13,"redis_duration_s":0.004578,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001001,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001359,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040434,"duration_s":0.048961,"completed_at":"2026-11-02T13:16:55.570Z","load_balancing_strategy":"primary","db_duration_s":0.005048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177gitlab # [ 406.717889] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7e1af1e339b7cd7d4c09e11","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7e1af1e339b7cd7d4c09e11: start","job_status":"start","queue_duration_s":1.282895,"scheduling_latency_s":1.282917}2178gitlab # [ 406.730295] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7e1af1e339b7cd7d4c09e11","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7e1af1e339b7cd7d4c09e11: done: 0.027051 sec","job_status":"done","queue_duration_s":1.282895,"scheduling_latency_s":1.282917,"redis_calls":10,"redis_duration_s":0.002201,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001271,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027051,"completed_at":"2026-11-02T13:16:55.599Z","load_balancing_strategy":"primary","db_duration_s":0.002573,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2179gitlab # [ 406.770134] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5df34637ecc39e792831d8ef","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5df34637ecc39e792831d8ef: start","job_status":"start","queue_duration_s":1.311819,"scheduling_latency_s":1.311854}2180gitlab # [ 406.782746] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5df34637ecc39e792831d8ef","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5df34637ecc39e792831d8ef: done: 0.034869 sec","job_status":"done","queue_duration_s":1.311819,"scheduling_latency_s":1.311854,"redis_calls":10,"redis_duration_s":0.002739,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001576,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034869,"completed_at":"2026-11-02T13:16:55.635Z","load_balancing_strategy":"primary","db_duration_s":0.003012,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2181gitlab # [ 406.822375] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"479d2f9ea4a65a913ed4ed8e","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-479d2f9ea4a65a913ed4ed8e: start","job_status":"start","queue_duration_s":1.349105,"scheduling_latency_s":1.349137}2182gitlab # [ 406.835118] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"479d2f9ea4a65a913ed4ed8e","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-479d2f9ea4a65a913ed4ed8e: done: 0.03339 sec","job_status":"done","queue_duration_s":1.349105,"scheduling_latency_s":1.349137,"redis_calls":10,"redis_duration_s":0.002379,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03339,"completed_at":"2026-11-02T13:16:55.671Z","load_balancing_strategy":"primary","db_duration_s":0.002496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183gitlab # [ 406.875725] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90382eb0a27e0b8b1835839d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90382eb0a27e0b8b1835839d: start","job_status":"start","queue_duration_s":1.385266,"scheduling_latency_s":1.385298}2184gitlab # [ 406.888450] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90382eb0a27e0b8b1835839d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90382eb0a27e0b8b1835839d: done: 0.034503 sec","job_status":"done","queue_duration_s":1.385266,"scheduling_latency_s":1.385298,"redis_calls":10,"redis_duration_s":0.003022,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034503,"completed_at":"2026-11-02T13:16:55.708Z","load_balancing_strategy":"primary","db_duration_s":0.00271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2185gitlab # [ 406.927826] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3a13d9482fed4268e5f2199","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3a13d9482fed4268e5f2199: start","job_status":"start","queue_duration_s":1.422476,"scheduling_latency_s":1.422511}2186gitlab # [ 406.940604] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3a13d9482fed4268e5f2199","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3a13d9482fed4268e5f2199: done: 0.034738 sec","job_status":"done","queue_duration_s":1.422476,"scheduling_latency_s":1.422511,"redis_calls":10,"redis_duration_s":0.0026040000000000004,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001476,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034738,"completed_at":"2026-11-02T13:16:55.746Z","load_balancing_strategy":"primary","db_duration_s":0.002752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187gitlab # [ 406.980279] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"40be62c01b78d7a6319513da","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-40be62c01b78d7a6319513da: start","job_status":"start","queue_duration_s":1.458938,"scheduling_latency_s":1.458966}2188gitlab # [ 406.992515] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"40be62c01b78d7a6319513da","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-40be62c01b78d7a6319513da: done: 0.025516 sec","job_status":"done","queue_duration_s":1.458938,"scheduling_latency_s":1.458966,"redis_calls":10,"redis_duration_s":0.00233,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001508,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025516,"completed_at":"2026-11-02T13:16:55.773Z","load_balancing_strategy":"primary","db_duration_s":0.002421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2189gitlab # [ 407.032066] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6d96b98fc2c0258d2b2ae893","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6d96b98fc2c0258d2b2ae893: start","job_status":"start","queue_duration_s":1.485894,"scheduling_latency_s":1.485915}2190gitlab # [ 407.044629] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6d96b98fc2c0258d2b2ae893","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6d96b98fc2c0258d2b2ae893: done: 0.03134 sec","job_status":"done","queue_duration_s":1.485894,"scheduling_latency_s":1.485915,"redis_calls":10,"redis_duration_s":0.002647,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03134,"completed_at":"2026-11-02T13:16:55.806Z","load_balancing_strategy":"primary","db_duration_s":0.003065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191gitlab # [ 407.084817] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a63c7a5be8510272c2f44e22","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a63c7a5be8510272c2f44e22: start","job_status":"start","queue_duration_s":1.519288,"scheduling_latency_s":1.519317}2192gitlab # [ 407.097420] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a63c7a5be8510272c2f44e22","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a63c7a5be8510272c2f44e22: done: 0.037773 sec","job_status":"done","queue_duration_s":1.519288,"scheduling_latency_s":1.519317,"redis_calls":10,"redis_duration_s":0.002595,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001463,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037773,"completed_at":"2026-11-02T13:16:55.846Z","load_balancing_strategy":"primary","db_duration_s":0.002885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193gitlab # [ 407.136957] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"991b58f8072b1887054dfc5f","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-991b58f8072b1887054dfc5f: start","job_status":"start","queue_duration_s":1.55903,"scheduling_latency_s":1.559059}2194gitlab # [ 407.149689] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"991b58f8072b1887054dfc5f","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-991b58f8072b1887054dfc5f: done: 0.031616 sec","job_status":"done","queue_duration_s":1.55903,"scheduling_latency_s":1.559059,"redis_calls":10,"redis_duration_s":0.00241,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001424,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031616,"completed_at":"2026-11-02T13:16:55.879Z","load_balancing_strategy":"primary","db_duration_s":0.002583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 407.189717] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"19cd1e942685584529974338","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-19cd1e942685584529974338: start","job_status":"start","queue_duration_s":1.592352,"scheduling_latency_s":1.592379}2196gitlab # [ 407.202366] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"19cd1e942685584529974338","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-19cd1e942685584529974338: done: 0.027075 sec","job_status":"done","queue_duration_s":1.592352,"scheduling_latency_s":1.592379,"redis_calls":10,"redis_duration_s":0.0021809999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027075,"completed_at":"2026-11-02T13:16:55.908Z","load_balancing_strategy":"primary","db_duration_s":0.002463,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197gitlab # [ 407.242202] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"26d6648b5aac2f08abc345a4","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-26d6648b5aac2f08abc345a4: start","job_status":"start","queue_duration_s":1.621122,"scheduling_latency_s":1.621151}2198gitlab # [ 407.254893] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"26d6648b5aac2f08abc345a4","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-26d6648b5aac2f08abc345a4: done: 0.026121 sec","job_status":"done","queue_duration_s":1.621122,"scheduling_latency_s":1.621151,"redis_calls":10,"redis_duration_s":0.0021279999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001246,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026121,"completed_at":"2026-11-02T13:16:55.936Z","load_balancing_strategy":"primary","db_duration_s":0.002396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199gitlab # [ 407.296075] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7371187fcebedf4e339816c5","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7371187fcebedf4e339816c5: start","job_status":"start","queue_duration_s":1.648949,"scheduling_latency_s":1.648979}2200gitlab # [ 407.308642] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7371187fcebedf4e339816c5","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7371187fcebedf4e339816c5: done: 0.025011 sec","job_status":"done","queue_duration_s":1.648949,"scheduling_latency_s":1.648979,"redis_calls":10,"redis_duration_s":0.001999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025011,"completed_at":"2026-11-02T13:16:55.963Z","load_balancing_strategy":"primary","db_duration_s":0.002645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201gitlab # [ 407.348431] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7d356075f75c410cedbf689e","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7d356075f75c410cedbf689e: start","job_status":"start","queue_duration_s":1.675593,"scheduling_latency_s":1.675621}2202gitlab # [ 407.361127] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7d356075f75c410cedbf689e","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7d356075f75c410cedbf689e: done: 0.024976 sec","job_status":"done","queue_duration_s":1.675593,"scheduling_latency_s":1.675621,"redis_calls":10,"redis_duration_s":0.00204,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024976,"completed_at":"2026-11-02T13:16:55.989Z","load_balancing_strategy":"primary","db_duration_s":0.002191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203gitlab # [ 407.400612] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:55.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7873828f7c138395dd37cd09","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7873828f7c138395dd37cd09: start","job_status":"start","queue_duration_s":1.70201,"scheduling_latency_s":1.702033}2204gitlab # [ 407.413191] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7873828f7c138395dd37cd09","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7873828f7c138395dd37cd09: done: 0.022052 sec","job_status":"done","queue_duration_s":1.70201,"scheduling_latency_s":1.702033,"redis_calls":10,"redis_duration_s":0.001781,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022052,"completed_at":"2026-11-02T13:16:56.013Z","load_balancing_strategy":"primary","db_duration_s":0.002343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205gitlab # [ 407.452576] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90e333d92d1533d7218640c5","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90e333d92d1533d7218640c5: start","job_status":"start","queue_duration_s":1.725464,"scheduling_latency_s":1.725486}2206gitlab # [ 407.466074] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"90e333d92d1533d7218640c5","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-90e333d92d1533d7218640c5: done: 0.022045 sec","job_status":"done","queue_duration_s":1.725464,"scheduling_latency_s":1.725486,"redis_calls":10,"redis_duration_s":0.00182,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022045,"completed_at":"2026-11-02T13:16:56.036Z","load_balancing_strategy":"primary","db_duration_s":0.002129,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2207gitlab # [ 407.505561] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d7d79621291f3fad2ce03032","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d7d79621291f3fad2ce03032: start","job_status":"start","queue_duration_s":1.749266,"scheduling_latency_s":1.749288}2208gitlab # [ 407.518136] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d7d79621291f3fad2ce03032","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d7d79621291f3fad2ce03032: done: 0.02182 sec","job_status":"done","queue_duration_s":1.749266,"scheduling_latency_s":1.749288,"redis_calls":10,"redis_duration_s":0.0018380000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02182,"completed_at":"2026-11-02T13:16:56.060Z","load_balancing_strategy":"primary","db_duration_s":0.002075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2210gitlab # [ 407.557868] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"97518d2fc5877469fb34a22f","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-97518d2fc5877469fb34a22f: start","job_status":"start","queue_duration_s":1.772474,"scheduling_latency_s":1.772495}2211gitlab # [ 407.569625] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"97518d2fc5877469fb34a22f","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-97518d2fc5877469fb34a22f: done: 0.023487 sec","job_status":"done","queue_duration_s":1.772474,"scheduling_latency_s":1.772495,"redis_calls":10,"redis_duration_s":0.002296,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001429,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023487,"completed_at":"2026-11-02T13:16:56.085Z","load_balancing_strategy":"primary","db_duration_s":0.002422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab # [ 407.608594] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5db0845d2aae3277b0e50e29","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5db0845d2aae3277b0e50e29: start","job_status":"start","queue_duration_s":1.797496,"scheduling_latency_s":1.797517}2213gitlab # [ 407.621228] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5db0845d2aae3277b0e50e29","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5db0845d2aae3277b0e50e29: done: 0.022829 sec","job_status":"done","queue_duration_s":1.797496,"scheduling_latency_s":1.797517,"redis_calls":10,"redis_duration_s":0.001806,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022829,"completed_at":"2026-11-02T13:16:56.109Z","load_balancing_strategy":"primary","db_duration_s":0.002324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214gitlab # [ 407.660864] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50061c601bb4f1db7fe1cac8","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50061c601bb4f1db7fe1cac8: start","job_status":"start","queue_duration_s":1.821821,"scheduling_latency_s":1.821843}2215gitlab # [ 407.673219] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50061c601bb4f1db7fe1cac8","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50061c601bb4f1db7fe1cac8: done: 0.023878 sec","job_status":"done","queue_duration_s":1.821821,"scheduling_latency_s":1.821843,"redis_calls":10,"redis_duration_s":0.0019259999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023878,"completed_at":"2026-11-02T13:16:56.134Z","load_balancing_strategy":"primary","db_duration_s":0.002419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 407.714083] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fbfcf0db8164f421f55b2c4d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fbfcf0db8164f421f55b2c4d: start","job_status":"start","queue_duration_s":1.847233,"scheduling_latency_s":1.847259}2217gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.17 seconds)2218gitlab # [ 407.726745] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fbfcf0db8164f421f55b2c4d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fbfcf0db8164f421f55b2c4d: done: 0.02489 sec","job_status":"done","queue_duration_s":1.847233,"scheduling_latency_s":1.847259,"redis_calls":10,"redis_duration_s":0.002007,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02489,"completed_at":"2026-11-02T13:16:56.161Z","load_balancing_strategy":"primary","db_duration_s":0.002436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2219gitlab # [ 407.766300] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cd0ec6a36e3e1ff2ea280fd3","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cd0ec6a36e3e1ff2ea280fd3: start","job_status":"start","queue_duration_s":1.873717,"scheduling_latency_s":1.873743}2220gitlab # [ 407.779248] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1222221gitlab # [ 407.782368] gitaly[920]: time="2026-11-02T13:16:56.662Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FRW74TMTD1570M3QVD1Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:26.560" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:56.660" grpc.time_ms=0.881 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2222gitlab # [ 407.795533] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cd0ec6a36e3e1ff2ea280fd3","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cd0ec6a36e3e1ff2ea280fd3: done: 0.025477 sec","job_status":"done","queue_duration_s":1.873717,"scheduling_latency_s":1.873743,"redis_calls":10,"redis_duration_s":0.002059,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001199,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025477,"completed_at":"2026-11-02T13:16:56.188Z","load_balancing_strategy":"primary","db_duration_s":0.00248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223gitlab # [ 407.834945] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6994c8adf35626006ccb4bd8","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6994c8adf35626006ccb4bd8: start","job_status":"start","queue_duration_s":1.901048,"scheduling_latency_s":1.901077}2224gitlab # [ 407.847598] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6994c8adf35626006ccb4bd8","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6994c8adf35626006ccb4bd8: done: 0.028665 sec","job_status":"done","queue_duration_s":1.901048,"scheduling_latency_s":1.901077,"redis_calls":10,"redis_duration_s":0.002187,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001262,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028665,"completed_at":"2026-11-02T13:16:56.218Z","load_balancing_strategy":"primary","db_duration_s":0.002658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2225gitlab # [ 407.887267] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ac778a2275ce1900ab2ee48d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ac778a2275ce1900ab2ee48d: start","job_status":"start","queue_duration_s":1.931421,"scheduling_latency_s":1.93145}2226gitlab # [ 407.901192] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ac778a2275ce1900ab2ee48d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ac778a2275ce1900ab2ee48d: done: 0.035547 sec","job_status":"done","queue_duration_s":1.931421,"scheduling_latency_s":1.93145,"redis_calls":10,"redis_duration_s":0.005013999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003198,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001816,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035547,"completed_at":"2026-11-02T13:16:56.256Z","load_balancing_strategy":"primary","db_duration_s":0.0028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2227gitlab # [ 407.941110] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"762b9016502286b59457779d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-762b9016502286b59457779d: start","job_status":"start","queue_duration_s":1.969282,"scheduling_latency_s":1.969321}2228gitlab # [ 407.953785] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"762b9016502286b59457779d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-762b9016502286b59457779d: done: 0.038256 sec","job_status":"done","queue_duration_s":1.969282,"scheduling_latency_s":1.969321,"redis_calls":10,"redis_duration_s":0.003592,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001701,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038256,"completed_at":"2026-11-02T13:16:56.296Z","load_balancing_strategy":"primary","db_duration_s":0.003328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229gitlab # [ 407.993137] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"32f9f7ae9efb3af7f94de785","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-32f9f7ae9efb3af7f94de785: start","job_status":"start","queue_duration_s":2.009557,"scheduling_latency_s":2.009594}2230gitlab # [ 408.005806] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"32f9f7ae9efb3af7f94de785","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-32f9f7ae9efb3af7f94de785: done: 0.02582 sec","job_status":"done","queue_duration_s":2.009557,"scheduling_latency_s":2.009594,"redis_calls":10,"redis_duration_s":0.002117,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02582,"completed_at":"2026-11-02T13:16:56.324Z","load_balancing_strategy":"primary","db_duration_s":0.002628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab # [ 408.045825] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"76666b0fd93edb9a62e38a1c","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-76666b0fd93edb9a62e38a1c: start","job_status":"start","queue_duration_s":2.037074,"scheduling_latency_s":2.037104}2232gitlab # [ 408.058622] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"76666b0fd93edb9a62e38a1c","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-76666b0fd93edb9a62e38a1c: done: 0.026314 sec","job_status":"done","queue_duration_s":2.037074,"scheduling_latency_s":2.037104,"redis_calls":10,"redis_duration_s":0.002231,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001282,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026314,"completed_at":"2026-11-02T13:16:56.352Z","load_balancing_strategy":"primary","db_duration_s":0.002619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233gitlab # [ 408.098529] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9f9abb4cbf9928b9d2431ab","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9f9abb4cbf9928b9d2431ab: start","job_status":"start","queue_duration_s":2.065089,"scheduling_latency_s":2.065117}2234gitlab # [ 408.112537] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9f9abb4cbf9928b9d2431ab","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9f9abb4cbf9928b9d2431ab: done: 0.031559 sec","job_status":"done","queue_duration_s":2.065089,"scheduling_latency_s":2.065117,"redis_calls":10,"redis_duration_s":0.0026490000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001611,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031559,"completed_at":"2026-11-02T13:16:56.385Z","load_balancing_strategy":"primary","db_duration_s":0.002814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2235gitlab # [ 408.152333] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ecba2e1802f421b7aed13d07","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ecba2e1802f421b7aed13d07: start","job_status":"start","queue_duration_s":2.099356,"scheduling_latency_s":2.099403}2236gitlab # [ 408.165034] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ecba2e1802f421b7aed13d07","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ecba2e1802f421b7aed13d07: done: 0.040376 sec","job_status":"done","queue_duration_s":2.099356,"scheduling_latency_s":2.099403,"redis_calls":10,"redis_duration_s":0.0031349999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040376,"completed_at":"2026-11-02T13:16:56.428Z","load_balancing_strategy":"primary","db_duration_s":0.003102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2237gitlab # [ 408.205095] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"23ef6b5a464f0039375719eb","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-23ef6b5a464f0039375719eb: start","job_status":"start","queue_duration_s":2.141961,"scheduling_latency_s":2.142}2238gitlab # [ 408.217715] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"23ef6b5a464f0039375719eb","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-23ef6b5a464f0039375719eb: done: 0.03228 sec","job_status":"done","queue_duration_s":2.141961,"scheduling_latency_s":2.142,"redis_calls":10,"redis_duration_s":0.002635,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001016,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001619,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03228,"completed_at":"2026-11-02T13:16:56.463Z","load_balancing_strategy":"primary","db_duration_s":0.002527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2239gitlab # [ 408.256833] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"134a90abc14117723ff10459","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-134a90abc14117723ff10459: start","job_status":"start","queue_duration_s":2.176072,"scheduling_latency_s":2.176102}2240gitlab # [ 408.269417] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"134a90abc14117723ff10459","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-134a90abc14117723ff10459: done: 0.026274 sec","job_status":"done","queue_duration_s":2.176072,"scheduling_latency_s":2.176102,"redis_calls":10,"redis_duration_s":0.0022040000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026274,"completed_at":"2026-11-02T13:16:56.491Z","load_balancing_strategy":"primary","db_duration_s":0.002448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 408.308972] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3c27d982cffa49a6a86e672","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3c27d982cffa49a6a86e672: start","job_status":"start","queue_duration_s":2.204062,"scheduling_latency_s":2.204089}2242gitlab # [ 408.323014] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3c27d982cffa49a6a86e672","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3c27d982cffa49a6a86e672: done: 0.025383 sec","job_status":"done","queue_duration_s":2.204062,"scheduling_latency_s":2.204089,"redis_calls":10,"redis_duration_s":0.002131,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025383,"completed_at":"2026-11-02T13:16:56.518Z","load_balancing_strategy":"primary","db_duration_s":0.002246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2243gitlab # [ 408.362657] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a8def70c15f767ff3a823ba4","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a8def70c15f767ff3a823ba4: start","job_status":"start","queue_duration_s":2.231175,"scheduling_latency_s":2.231201}2244gitlab # [ 408.375323] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a8def70c15f767ff3a823ba4","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a8def70c15f767ff3a823ba4: done: 0.028218 sec","job_status":"done","queue_duration_s":2.231175,"scheduling_latency_s":2.231201,"redis_calls":10,"redis_duration_s":0.002353,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028218,"completed_at":"2026-11-02T13:16:56.548Z","load_balancing_strategy":"primary","db_duration_s":0.002584,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 408.415177] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"69943b8fa2b495e5ba5838f2","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-69943b8fa2b495e5ba5838f2: start","job_status":"start","queue_duration_s":2.261209,"scheduling_latency_s":2.261239}2246gitlab # [ 408.427989] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"69943b8fa2b495e5ba5838f2","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-69943b8fa2b495e5ba5838f2: done: 0.042043 sec","job_status":"done","queue_duration_s":2.261209,"scheduling_latency_s":2.261239,"redis_calls":10,"redis_duration_s":0.005672,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004476,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042043,"completed_at":"2026-11-02T13:16:56.592Z","load_balancing_strategy":"primary","db_duration_s":0.004815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2247gitlab # [ 408.467621] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f3d58e9e566d989880fa2d1","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f3d58e9e566d989880fa2d1: start","job_status":"start","queue_duration_s":2.305769,"scheduling_latency_s":2.30581}2248gitlab # [ 408.480434] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f3d58e9e566d989880fa2d1","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f3d58e9e566d989880fa2d1: done: 0.043735 sec","job_status":"done","queue_duration_s":2.305769,"scheduling_latency_s":2.30581,"redis_calls":10,"redis_duration_s":0.00331,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043735,"completed_at":"2026-11-02T13:16:56.638Z","load_balancing_strategy":"primary","db_duration_s":0.005861,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2249gitlab # [ 408.520199] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e78ab4b257aa9f1891050ee","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e78ab4b257aa9f1891050ee: start","job_status":"start","queue_duration_s":2.351991,"scheduling_latency_s":2.352032}2250gitlab # [ 408.534020] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e78ab4b257aa9f1891050ee","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e78ab4b257aa9f1891050ee: done: 0.043526 sec","job_status":"done","queue_duration_s":2.351991,"scheduling_latency_s":2.352032,"redis_calls":10,"redis_duration_s":0.003353,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00183,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043526,"completed_at":"2026-11-02T13:16:56.684Z","load_balancing_strategy":"primary","db_duration_s":0.00475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251gitlab # [ 408.573398] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f672432120ebb33c5733bea4","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f672432120ebb33c5733bea4: start","job_status":"start","queue_duration_s":2.397451,"scheduling_latency_s":2.397482}2252gitlab # [ 408.586176] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f672432120ebb33c5733bea4","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f672432120ebb33c5733bea4: done: 0.033409 sec","job_status":"done","queue_duration_s":2.397451,"scheduling_latency_s":2.397482,"redis_calls":10,"redis_duration_s":0.004345,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001693,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033409,"completed_at":"2026-11-02T13:16:56.720Z","load_balancing_strategy":"primary","db_duration_s":0.004975,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2253gitlab # [ 408.625958] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"98bc49dcd1caabe5de4faa47","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-98bc49dcd1caabe5de4faa47: start","job_status":"start","queue_duration_s":2.43256,"scheduling_latency_s":2.43259}2254gitlab # [ 408.638579] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"98bc49dcd1caabe5de4faa47","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-98bc49dcd1caabe5de4faa47: done: 0.033641 sec","job_status":"done","queue_duration_s":2.43256,"scheduling_latency_s":2.43259,"redis_calls":10,"redis_duration_s":0.005595,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004469,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033641,"completed_at":"2026-11-02T13:16:56.755Z","load_balancing_strategy":"primary","db_duration_s":0.003916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255gitlab # [ 408.678493] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c852959c73449e685b811d14","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c852959c73449e685b811d14: start","job_status":"start","queue_duration_s":2.467995,"scheduling_latency_s":2.468023}2256gitlab # [ 408.691329] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c852959c73449e685b811d14","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c852959c73449e685b811d14: done: 0.030326 sec","job_status":"done","queue_duration_s":2.467995,"scheduling_latency_s":2.468023,"redis_calls":10,"redis_duration_s":0.002448,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030326,"completed_at":"2026-11-02T13:16:56.787Z","load_balancing_strategy":"primary","db_duration_s":0.002939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2258gitlab # [ 408.730938] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4882e353ed488d2b3823e5d2","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4882e353ed488d2b3823e5d2: start","job_status":"start","queue_duration_s":2.500168,"scheduling_latency_s":2.500197}2259gitlab # [ 408.744283] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4882e353ed488d2b3823e5d2","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4882e353ed488d2b3823e5d2: done: 0.033477 sec","job_status":"done","queue_duration_s":2.500168,"scheduling_latency_s":2.500197,"redis_calls":10,"redis_duration_s":0.0030150000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033477,"completed_at":"2026-11-02T13:16:56.822Z","load_balancing_strategy":"primary","db_duration_s":0.002973,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2260gitlab # [ 408.784490] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a66346b205e48cc66485a4fa","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a66346b205e48cc66485a4fa: start","job_status":"start","queue_duration_s":2.535607,"scheduling_latency_s":2.535641}2261gitlab # [ 408.797174] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a66346b205e48cc66485a4fa","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a66346b205e48cc66485a4fa: done: 0.031082 sec","job_status":"done","queue_duration_s":2.535607,"scheduling_latency_s":2.535641,"redis_calls":10,"redis_duration_s":0.002451,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00154,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031082,"completed_at":"2026-11-02T13:16:56.855Z","load_balancing_strategy":"primary","db_duration_s":0.002727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262gitlab # [ 408.836724] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b7eca4ba3f8f29e36d36ff66","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b7eca4ba3f8f29e36d36ff66: start","job_status":"start","queue_duration_s":2.568138,"scheduling_latency_s":2.568164}2263gitlab # [ 408.849294] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b7eca4ba3f8f29e36d36ff66","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b7eca4ba3f8f29e36d36ff66: done: 0.023167 sec","job_status":"done","queue_duration_s":2.568138,"scheduling_latency_s":2.568164,"redis_calls":10,"redis_duration_s":0.0021200000000000004,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00128,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023167,"completed_at":"2026-11-02T13:16:56.880Z","load_balancing_strategy":"primary","db_duration_s":0.002516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2264gitlab # [ 408.888570] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"055d443daa04a7677268c55d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-055d443daa04a7677268c55d: start","job_status":"start","queue_duration_s":2.592946,"scheduling_latency_s":2.592971}2265gitlab # [ 408.901481] nix-daemon[3565]: accepted connection from pid 4655, user nixbot2266gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.18 seconds)2267gitlab # [ 408.902790] gitaly[920]: time="2026-11-02T13:16:57.846Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FT12MTJ09PMBG649DZNZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:27.442" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:57.843" grpc.time_ms=3.052 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2268gitlab # [ 408.916306] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"055d443daa04a7677268c55d","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-055d443daa04a7677268c55d: done: 0.024483 sec","job_status":"done","queue_duration_s":2.592946,"scheduling_latency_s":2.592971,"redis_calls":10,"redis_duration_s":0.002135,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024483,"completed_at":"2026-11-02T13:16:56.906Z","load_balancing_strategy":"primary","db_duration_s":0.002438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2269gitlab # [ 408.956666] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"886c72e01fa1be4337f37a53","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-886c72e01fa1be4337f37a53: start","job_status":"start","queue_duration_s":2.619155,"scheduling_latency_s":2.619183}2270gitlab # [ 408.969731] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1232271gitlab # [ 408.972491] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"886c72e01fa1be4337f37a53","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-886c72e01fa1be4337f37a53: done: 0.025385 sec","job_status":"done","queue_duration_s":2.619155,"scheduling_latency_s":2.619183,"redis_calls":10,"redis_duration_s":0.002341,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001392,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025385,"completed_at":"2026-11-02T13:16:56.933Z","load_balancing_strategy":"primary","db_duration_s":0.002565,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2272gitlab # [ 409.012095] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"35bf49eedb9bf681bcde7edf","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-35bf49eedb9bf681bcde7edf: start","job_status":"start","queue_duration_s":2.646243,"scheduling_latency_s":2.646273}2273gitlab # [ 409.024330] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"35bf49eedb9bf681bcde7edf","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-35bf49eedb9bf681bcde7edf: done: 0.024699 sec","job_status":"done","queue_duration_s":2.646243,"scheduling_latency_s":2.646273,"redis_calls":10,"redis_duration_s":0.002287,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001365,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024699,"completed_at":"2026-11-02T13:16:56.960Z","load_balancing_strategy":"primary","db_duration_s":0.002014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2274gitlab # [ 409.063984] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2674e579618d82a2270d64f6","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2674e579618d82a2270d64f6: start","job_status":"start","queue_duration_s":2.672531,"scheduling_latency_s":2.67256}2275gitlab # [ 409.076499] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2674e579618d82a2270d64f6","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2674e579618d82a2270d64f6: done: 0.024869 sec","job_status":"done","queue_duration_s":2.672531,"scheduling_latency_s":2.67256,"redis_calls":10,"redis_duration_s":0.002014,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024869,"completed_at":"2026-11-02T13:16:56.986Z","load_balancing_strategy":"primary","db_duration_s":0.00229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 409.116391] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:56.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9ad466bfa5b5a798a1a3140","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9ad466bfa5b5a798a1a3140: start","job_status":"start","queue_duration_s":2.698991,"scheduling_latency_s":2.699019}2277gitlab # [ 409.128950] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9ad466bfa5b5a798a1a3140","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9ad466bfa5b5a798a1a3140: done: 0.025117 sec","job_status":"done","queue_duration_s":2.698991,"scheduling_latency_s":2.699019,"redis_calls":10,"redis_duration_s":0.0020670000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025117,"completed_at":"2026-11-02T13:16:57.013Z","load_balancing_strategy":"primary","db_duration_s":0.002296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2278gitlab # [ 409.169781] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b5d7879d73bc01fa8fb30003","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b5d7879d73bc01fa8fb30003: start","job_status":"start","queue_duration_s":2.725828,"scheduling_latency_s":2.725856}2279gitlab # [ 409.182427] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b5d7879d73bc01fa8fb30003","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b5d7879d73bc01fa8fb30003: done: 0.025007 sec","job_status":"done","queue_duration_s":2.725828,"scheduling_latency_s":2.725856,"redis_calls":10,"redis_duration_s":0.002163,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001314,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025007,"completed_at":"2026-11-02T13:16:57.040Z","load_balancing_strategy":"primary","db_duration_s":0.002424,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2280gitlab # [ 409.222268] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4310b4a03ce4ccbaa2273277","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4310b4a03ce4ccbaa2273277: start","job_status":"start","queue_duration_s":2.752604,"scheduling_latency_s":2.752634}2281gitlab # [ 409.234993] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4310b4a03ce4ccbaa2273277","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4310b4a03ce4ccbaa2273277: done: 0.024551 sec","job_status":"done","queue_duration_s":2.752604,"scheduling_latency_s":2.752634,"redis_calls":10,"redis_duration_s":0.002149,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024551,"completed_at":"2026-11-02T13:16:57.066Z","load_balancing_strategy":"primary","db_duration_s":0.002133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2282gitlab # [ 409.274799] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"193391bd5fbc3cbf099f94ef","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-193391bd5fbc3cbf099f94ef: start","job_status":"start","queue_duration_s":2.778918,"scheduling_latency_s":2.778945}2283gitlab # [ 409.287586] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"193391bd5fbc3cbf099f94ef","created_at":"2026-11-02T13:16:54.184Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-0ad66e615da742e6","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"50061428bbc18533f28a1e678f29f89c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.289Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-193391bd5fbc3cbf099f94ef: done: 0.027959 sec","job_status":"done","queue_duration_s":2.778918,"scheduling_latency_s":2.778945,"redis_calls":10,"redis_duration_s":0.0025250000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001445,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027959,"completed_at":"2026-11-02T13:16:57.096Z","load_balancing_strategy":"primary","db_duration_s":0.002763,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2284gitlab # [ 409.327345] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.098Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"670e9aae64eec6631f79c03d","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-670e9aae64eec6631f79c03d: start","job_status":"start","queue_duration_s":2.502493,"scheduling_latency_s":2.502527}2285gitlab # [ 409.340216] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.126Z","message":"No existing merge request to be cleaned up."}2286gitlab # [ 409.341922] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.150Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"670e9aae64eec6631f79c03d","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-670e9aae64eec6631f79c03d: done: 0.052091 sec","job_status":"done","queue_duration_s":2.502493,"scheduling_latency_s":2.502527,"redis_calls":12,"redis_duration_s":0.0042959999999999995,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000864,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00174,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052091,"completed_at":"2026-11-02T13:16:57.150Z","load_balancing_strategy":"primary","db_duration_s":0.005733,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2287gitlab # [ 409.384191] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.152Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ed249a4e3534a0566f06434","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ed249a4e3534a0566f06434: start","job_status":"start","queue_duration_s":2.556969,"scheduling_latency_s":2.557009}2288gitlab # [ 409.396828] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.173Z","message":"No existing merge request to be cleaned up."}2289gitlab # [ 409.398534] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.189Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ed249a4e3534a0566f06434","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ed249a4e3534a0566f06434: done: 0.036767 sec","job_status":"done","queue_duration_s":2.556969,"scheduling_latency_s":2.557009,"redis_calls":10,"redis_duration_s":0.002903,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001637,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036767,"completed_at":"2026-11-02T13:16:57.189Z","load_balancing_strategy":"primary","db_duration_s":0.002837,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2290gitlab # [ 409.438108] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.191Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bac8ebe865448b105db95256","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bac8ebe865448b105db95256: start","job_status":"start","queue_duration_s":2.595467,"scheduling_latency_s":2.595496}2291gitlab # [ 409.450793] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.205Z","message":"No existing merge request to be cleaned up."}2292gitlab # [ 409.452549] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.216Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bac8ebe865448b105db95256","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bac8ebe865448b105db95256: done: 0.025757 sec","job_status":"done","queue_duration_s":2.595467,"scheduling_latency_s":2.595496,"redis_calls":10,"redis_duration_s":0.002144,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001225,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025757,"completed_at":"2026-11-02T13:16:57.216Z","load_balancing_strategy":"primary","db_duration_s":0.002288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2293gitlab # [ 409.492165] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.218Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1b3914832d3ff9c3095557d1","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1b3914832d3ff9c3095557d1: start","job_status":"start","queue_duration_s":2.622778,"scheduling_latency_s":2.622803}2294gitlab # [ 409.504639] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.232Z","message":"No existing merge request to be cleaned up."}2295gitlab # [ 409.506410] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.245Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1b3914832d3ff9c3095557d1","created_at":"2026-11-02T13:16:54.580Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-91a670ee6b24435d","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"20f18dffc35cb46b92592793fa193c64","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:54.595Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1b3914832d3ff9c3095557d1: done: 0.026587 sec","job_status":"done","queue_duration_s":2.622778,"scheduling_latency_s":2.622803,"redis_calls":10,"redis_duration_s":0.002164,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00122,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026587,"completed_at":"2026-11-02T13:16:57.245Z","load_balancing_strategy":"primary","db_duration_s":0.002239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296gitlab # [ 409.546293] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49db62cd0f4c9eb1e89c18aa","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49db62cd0f4c9eb1e89c18aa: start","job_status":"start","queue_duration_s":1.797364,"scheduling_latency_s":1.797391}2297gitlab # [ 409.559009] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49db62cd0f4c9eb1e89c18aa","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49db62cd0f4c9eb1e89c18aa: done: 0.028503 sec","job_status":"done","queue_duration_s":1.797364,"scheduling_latency_s":1.797391,"redis_calls":15,"redis_duration_s":0.0030110000000000002,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000478,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001662,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028503,"completed_at":"2026-11-02T13:16:57.275Z","load_balancing_strategy":"primary","db_duration_s":0.000588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2298gitlab # [ 409.602345] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b758db41fb92a5d7241b629","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b758db41fb92a5d7241b629: start","job_status":"start","queue_duration_s":1.827526,"scheduling_latency_s":1.827554}2299gitlab # [ 409.615228] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b758db41fb92a5d7241b629","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b758db41fb92a5d7241b629: done: 0.022123 sec","job_status":"done","queue_duration_s":1.827526,"scheduling_latency_s":1.827554,"redis_calls":13,"redis_duration_s":0.002835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022123,"completed_at":"2026-11-02T13:16:57.299Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2300gitlab # [ 409.655589] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ad76246b04df6184b52751","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ad76246b04df6184b52751: start","job_status":"start","queue_duration_s":1.852921,"scheduling_latency_s":1.852962}2301gitlab # [ 409.668494] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ad76246b04df6184b52751","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ad76246b04df6184b52751: done: 0.033198 sec","job_status":"done","queue_duration_s":1.852921,"scheduling_latency_s":1.852962,"redis_calls":13,"redis_duration_s":0.0033369999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033198,"completed_at":"2026-11-02T13:16:57.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2302gitlab # [ 409.709064] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e7f4798c80e73b2396fdfab","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e7f4798c80e73b2396fdfab: start","job_status":"start","queue_duration_s":1.888542,"scheduling_latency_s":1.888582}2303gitlab # [ 409.721892] gitaly[920]: time="2026-11-02T13:16:58.610Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:28.208" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:58.609" grpc.time_ms=0.791 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2304gitlab # [ 409.735082] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e7f4798c80e73b2396fdfab","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e7f4798c80e73b2396fdfab: done: 0.028602 sec","job_status":"done","queue_duration_s":1.888542,"scheduling_latency_s":1.888582,"redis_calls":13,"redis_duration_s":0.002879,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001849,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028602,"completed_at":"2026-11-02T13:16:57.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2305gitlab # [ 409.775837] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9196548cadae88ef36a0c954","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9196548cadae88ef36a0c954: start","job_status":"start","queue_duration_s":1.918903,"scheduling_latency_s":1.91893}2306gitlab # [ 409.788913] gitaly[920]: time="2026-11-02T13:16:58.696Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:17:08.262" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:58.682" grpc.time_ms=13.872 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2307gitlab # [ 409.803747] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 2272308gitlab # [ 409.806441] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9196548cadae88ef36a0c954","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9196548cadae88ef36a0c954: done: 0.019865 sec","job_status":"done","queue_duration_s":1.918903,"scheduling_latency_s":1.91893,"redis_calls":13,"redis_duration_s":0.002701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001828,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019865,"completed_at":"2026-11-02T13:16:57.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2309gitlab # [ 409.847052] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0af05ff8c9f111ddfa9cd54","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0af05ff8c9f111ddfa9cd54: start","job_status":"start","queue_duration_s":1.940408,"scheduling_latency_s":1.940433}2310gitlab # [ 409.860010] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0af05ff8c9f111ddfa9cd54","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0af05ff8c9f111ddfa9cd54: done: 0.018994 sec","job_status":"done","queue_duration_s":1.940408,"scheduling_latency_s":1.940433,"redis_calls":13,"redis_duration_s":0.002419,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001631,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018994,"completed_at":"2026-11-02T13:16:57.408Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2311gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2312gitlab # [ 409.900229] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0fc34305fa7fd0d21ae8f1a","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0fc34305fa7fd0d21ae8f1a: start","job_status":"start","queue_duration_s":1.961009,"scheduling_latency_s":1.961032}2313gitlab # [ 409.913208] gitaly[920]: time="2026-11-02T13:16:58.824Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FV0JM2MHB7HNA10Y3THY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:28.423" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:58.823" grpc.time_ms=0.681 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2314gitlab # [ 409.925873] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0fc34305fa7fd0d21ae8f1a","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0fc34305fa7fd0d21ae8f1a: done: 0.017921 sec","job_status":"done","queue_duration_s":1.961009,"scheduling_latency_s":1.961032,"redis_calls":13,"redis_duration_s":0.002054,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017921,"completed_at":"2026-11-02T13:16:57.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2315gitlab # [ 409.966475] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cf8972103ca4155d0cf6930","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cf8972103ca4155d0cf6930: start","job_status":"start","queue_duration_s":1.980667,"scheduling_latency_s":1.980698}2316gitlab # [ 409.979420] gitaly[920]: time="2026-11-02T13:16:58.869Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FV0JM2MHB7HNA10Y3THY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:17:08.210" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:58.860" grpc.time_ms=8.27 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2317gitlab # [ 409.993869] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1592318gitlab # [ 409.996432] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cf8972103ca4155d0cf6930","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cf8972103ca4155d0cf6930: done: 0.388391 sec","job_status":"done","queue_duration_s":1.980667,"scheduling_latency_s":1.980698,"redis_calls":13,"redis_duration_s":0.099525,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.09727,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002255,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.388391,"completed_at":"2026-11-02T13:16:57.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2319gitlab # [ 410.037520] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca48a5bc689c8a2e46378ee6","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca48a5bc689c8a2e46378ee6: start","job_status":"start","queue_duration_s":2.385403,"scheduling_latency_s":2.385454}2320gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.17 seconds)2321gitlab # [ 410.050650] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca48a5bc689c8a2e46378ee6","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca48a5bc689c8a2e46378ee6: done: 0.041362 sec","job_status":"done","queue_duration_s":2.385403,"scheduling_latency_s":2.385454,"redis_calls":13,"redis_duration_s":0.006294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003762,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041362,"completed_at":"2026-11-02T13:16:57.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2322gitlab # [ 410.091121] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a34b75246f37427b35edca8e","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a34b75246f37427b35edca8e: start","job_status":"start","queue_duration_s":2.428582,"scheduling_latency_s":2.428616}2323gitlab # [ 410.103933] gitaly[920]: time="2026-11-02T13:16:59.017Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FV62FES9E3FXX7PWAW2P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:29.015" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:59.015" grpc.time_ms=0.942 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2324gitlab # [ 410.117010] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses HTTP/1.1" 200 1314 "" "curl/8.21.0" 1082325gitlab # [ 410.119683] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a34b75246f37427b35edca8e","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a34b75246f37427b35edca8e: done: 0.024809 sec","job_status":"done","queue_duration_s":2.428582,"scheduling_latency_s":2.428616,"redis_calls":13,"redis_duration_s":0.004001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001171,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00283,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024809,"completed_at":"2026-11-02T13:16:57.902Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2326gitlab # [ 410.160738] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98bcf5df4c96ec6273653d83","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98bcf5df4c96ec6273653d83: start","job_status":"start","queue_duration_s":2.456491,"scheduling_latency_s":2.456526}2327gitlab # [ 410.173523] gitaly[920]: time="2026-11-02T13:16:59.020Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FV63QXC38E99BCKXTQCR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:29.017" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:59.018" grpc.time_ms=2.266 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2328gitlab # [ 410.186654] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5bcc93bc627dd5c1ed48f56f948c7335c5310516 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1762329gitlab # [ 410.189329] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98bcf5df4c96ec6273653d83","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98bcf5df4c96ec6273653d83: done: 0.022011 sec","job_status":"done","queue_duration_s":2.456491,"scheduling_latency_s":2.456526,"redis_calls":13,"redis_duration_s":0.002358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001349,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022011,"completed_at":"2026-11-02T13:16:57.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2330gitlab # [ 410.230717] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9bca19e0f907a3febd5199c","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9bca19e0f907a3febd5199c: start","job_status":"start","queue_duration_s":2.480045,"scheduling_latency_s":2.480077}2331gitlab # [ 410.243627] gitaly[920]: time="2026-11-02T13:16:59.035Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-02T13:17:29.033" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:59.034" grpc.time_ms=0.67 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2332gitlab # [ 410.256911] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9bca19e0f907a3febd5199c","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9bca19e0f907a3febd5199c: done: 0.020609 sec","job_status":"done","queue_duration_s":2.480045,"scheduling_latency_s":2.480077,"redis_calls":13,"redis_duration_s":0.002224,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001344,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020609,"completed_at":"2026-11-02T13:16:57.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2333gitlab # [ 410.297025] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"004ff36ab40b670a06927bf6","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-004ff36ab40b670a06927bf6: start","job_status":"start","queue_duration_s":2.502367,"scheduling_latency_s":2.502399}2334gitlab # [ 410.309741] gitaly[920]: time="2026-11-02T13:16:59.070Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FV63QXC38E99BCKXTQCR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-02T13:17:09.041" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:59.062" grpc.time_ms=7.811 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2335gitlab # [ 410.323444] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"004ff36ab40b670a06927bf6","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-004ff36ab40b670a06927bf6: done: 0.022282 sec","job_status":"done","queue_duration_s":2.502367,"scheduling_latency_s":2.502399,"redis_calls":13,"redis_duration_s":0.002262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001469,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022282,"completed_at":"2026-11-02T13:16:57.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2336gitlab # [ 410.363776] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9305d625c19dd204ebcdc82","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9305d625c19dd204ebcdc82: start","job_status":"start","queue_duration_s":2.52628,"scheduling_latency_s":2.52631}2337gitlab # [ 410.376642] gitaly[920]: time="2026-11-02T13:16:59.085Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:29.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:59.084" grpc.time_ms=0.769 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2338gitlab # [ 410.390158] gitaly[920]: time="2026-11-02T13:16:59.306Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:59.305" grpc.time_ms=0.614 pid=920 span.kind=server system=grpc2339gitlab # [ 410.400125] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9305d625c19dd204ebcdc82","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9305d625c19dd204ebcdc82: done: 0.020337 sec","job_status":"done","queue_duration_s":2.52628,"scheduling_latency_s":2.52631,"redis_calls":13,"redis_duration_s":0.002702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001491,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020337,"completed_at":"2026-11-02T13:16:57.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2340gitlab # [ 410.442005] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:57.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3a9191b8f2f38d5be202aa2","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3a9191b8f2f38d5be202aa2: start","job_status":"start","queue_duration_s":2.548046,"scheduling_latency_s":2.548071}2341gitlab # [ 410.455202] nixbot[3407]: {"ts": "2026-11-02T13:16:59.381Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "5bcc93bc627dd5c1ed48f56f948c7335c5310516", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}2342gitlab # [ 410.458726] gitaly[920]: time="2026-11-02T13:16:59.334Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:59.333" grpc.time_ms=0.663 pid=920 span.kind=server system=grpc2343gitlab # [ 410.468635] gitaly[920]: time="2026-11-02T13:16:59.357Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-02T13:16:59.356" grpc.time_ms=0.432 pid=920 span.kind=server system=grpc2344gitlab # [ 410.478454] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3a9191b8f2f38d5be202aa2","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3a9191b8f2f38d5be202aa2: done: 0.018816 sec","job_status":"done","queue_duration_s":2.548046,"scheduling_latency_s":2.548071,"redis_calls":13,"redis_duration_s":0.002071,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018816,"completed_at":"2026-11-02T13:16:58.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2345gitlab # [ 410.518302] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43e49a9718c73cebb4ab6a4e","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43e49a9718c73cebb4ab6a4e: start","job_status":"start","queue_duration_s":2.56832,"scheduling_latency_s":2.568346}2346gitlab # [ 410.531079] gitaly[920]: time="2026-11-02T13:16:59.360Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:29.270" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:59.270" operations="[repack_refs]" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2347gitlab # [ 410.541204] gitaly[920]: time="2026-11-02T13:16:59.363Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:29.270" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:59.270" pid=920 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-11-02 13:15:07.95799756 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2348gitlab # [ 410.553374] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 632349gitlab # [ 410.555553] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43e49a9718c73cebb4ab6a4e","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43e49a9718c73cebb4ab6a4e: done: 0.018836 sec","job_status":"done","queue_duration_s":2.56832,"scheduling_latency_s":2.568346,"redis_calls":13,"redis_duration_s":0.0023090000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001502,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018836,"completed_at":"2026-11-02T13:16:58.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2350gitlab # [ 410.596371] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4138b343695d97d30686caa","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4138b343695d97d30686caa: start","job_status":"start","queue_duration_s":2.589896,"scheduling_latency_s":2.589923}2351gitlab # [ 410.608959] gitaly[920]: time="2026-11-02T13:16:59.366Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY diskcache=4762dad8-fe4c-422f-8019-95be32b40e38 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:29.270" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:59.270" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2352gitlab # [ 410.618970] gitaly[920]: time="2026-11-02T13:16:59.366Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=82 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7245 command.oublock=0 command.real_time_ms=88 command.spawn_token_fork_ms=0 command.system_time_ms=52 command.user_time_ms=30 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:29.270" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:59.270" grpc.time_ms=95.859 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2353gitlab # [ 410.632685] gitlab-workhorse[826]: time="2026-11-02T13:16:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672354gitlab # [ 410.634610] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 632355gitlab # [ 410.637639] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4138b343695d97d30686caa","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4138b343695d97d30686caa: done: 0.019537 sec","job_status":"done","queue_duration_s":2.589896,"scheduling_latency_s":2.589923,"redis_calls":13,"redis_duration_s":0.002418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001609,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019537,"completed_at":"2026-11-02T13:16:58.058Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2356gitlab # [ 410.677659] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397864b5843d9daa72570ad7","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397864b5843d9daa72570ad7: start","job_status":"start","queue_duration_s":2.610912,"scheduling_latency_s":2.610938}2357gitlab # [ 410.690667] gitaly[920]: time="2026-11-02T13:16:59.390Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:29.270" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:59.270" pid=920 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2358gitlab # [ 410.704402] gitaly[920]: time="2026-11-02T13:16:59.390Z" level=info msg="optimized repository" clean_stale_data_time=27.13906ms clean_work_trees_time="104.22µs" commit_graph_time=360ns correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.4µs" pid=920 prune_time=260ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="1.1µs" storage_name=default system=housekeeping total_time=27.29166ms2359gitlab # [ 410.710415] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 672360gitlab # [ 410.712545] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397864b5843d9daa72570ad7","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397864b5843d9daa72570ad7: done: 0.019412 sec","job_status":"done","queue_duration_s":2.610912,"scheduling_latency_s":2.610938,"redis_calls":13,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000761,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001544,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019412,"completed_at":"2026-11-02T13:16:58.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2361gitlab # [ 410.752717] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f9407188a4dcd314ea039a5","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f9407188a4dcd314ea039a5: start","job_status":"start","queue_duration_s":2.631802,"scheduling_latency_s":2.631829}2362gitlab # [ 410.765409] gitaly[920]: time="2026-11-02T13:16:59.390Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-02T13:17:29.270" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-02T13:16:59.270" pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2363gitlab # [ 410.775114] gitaly[920]: time="2026-11-02T13:16:59.436Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-02T13:17:29.435" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:59.435" grpc.time_ms=0.716 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2364gitlab # [ 410.788293] gitlab-workhorse[826]: time="2026-11-02T13:16:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672365gitlab # [ 410.790190] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/11/02:13:16:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 782366gitlab # [ 410.792132] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f9407188a4dcd314ea039a5","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f9407188a4dcd314ea039a5: done: 0.018777 sec","job_status":"done","queue_duration_s":2.631802,"scheduling_latency_s":2.631829,"redis_calls":13,"redis_duration_s":0.002213,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001475,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018777,"completed_at":"2026-11-02T13:16:58.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2367gitlab # [ 410.832207] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50dc655d82307822cca0c02f","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50dc655d82307822cca0c02f: start","job_status":"start","queue_duration_s":2.65193,"scheduling_latency_s":2.651953}2368gitlab # [ 410.844896] gitaly[920]: time="2026-11-02T13:16:59.475Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FTS5TSKG3J93MWVCRGQY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-02T13:17:29.473" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-02T13:16:59.473" grpc.time_ms=1.469 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2369gitlab # [ 410.858711] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50dc655d82307822cca0c02f","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50dc655d82307822cca0c02f: done: 0.019075 sec","job_status":"done","queue_duration_s":2.65193,"scheduling_latency_s":2.651953,"redis_calls":13,"redis_duration_s":0.0020020000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019075,"completed_at":"2026-11-02T13:16:58.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2370gitlab # [ 410.899089] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d03484df1568ac83fdfd868","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d03484df1568ac83fdfd868: start","job_status":"start","queue_duration_s":2.672533,"scheduling_latency_s":2.672557}2371gitlab # [ 410.912045] gitaly[920]: time="2026-11-02T13:16:59.482Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FVMWBYGSFQJR5ZVHRQ0H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:59.473" grpc.time_ms=8.995 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2372gitlab # [ 410.925619] gitaly[920]: time="2026-11-02T13:16:59.547Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FVPW6KHX4JBY5ER0SCPP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:59.537" pid=920 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2373gitlab # [ 410.936086] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d03484df1568ac83fdfd868","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d03484df1568ac83fdfd868: done: 0.018967 sec","job_status":"done","queue_duration_s":2.672533,"scheduling_latency_s":2.672557,"redis_calls":13,"redis_duration_s":0.002284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001517,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018967,"completed_at":"2026-11-02T13:16:58.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2374gitlab # [ 410.976161] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"faba98b18189008b91cbe246","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-faba98b18189008b91cbe246: start","job_status":"start","queue_duration_s":2.692928,"scheduling_latency_s":2.692953}2375gitlab # [ 410.988831] gitaly[920]: time="2026-11-02T13:16:59.547Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FVPW6KHX4JBY5ER0SCPP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:59.537" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2376gitlab # [ 411.002973] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"faba98b18189008b91cbe246","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-faba98b18189008b91cbe246: done: 0.019084 sec","job_status":"done","queue_duration_s":2.692928,"scheduling_latency_s":2.692953,"redis_calls":13,"redis_duration_s":0.002222,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019084,"completed_at":"2026-11-02T13:16:58.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2377gitlab # [ 411.043570] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e627027d4625a418da802c34","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e627027d4625a418da802c34: start","job_status":"start","queue_duration_s":2.713499,"scheduling_latency_s":2.713523}2378gitlab # [ 411.056536] gitaly[920]: time="2026-11-02T13:16:59.548Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FVPW6KHX4JBY5ER0SCPP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:59.537" grpc.time_ms=10.062 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2379gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses2380gitlab # [ 411.072475] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e627027d4625a418da802c34","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e627027d4625a418da802c34: done: 0.020296 sec","job_status":"done","queue_duration_s":2.713499,"scheduling_latency_s":2.713523,"redis_calls":13,"redis_duration_s":0.002211,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020296,"completed_at":"2026-11-02T13:16:58.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2381gitlab # [ 411.111300] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42d099c3c744d3686b401b08","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42d099c3c744d3686b401b08: start","job_status":"start","queue_duration_s":2.7353,"scheduling_latency_s":2.735324}2382gitlab # [ 411.124193] gitaly[920]: time="2026-11-02T13:16:59.656Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M6E6FVT4YFK75EHQ8YTZFEBH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:59.647" grpc.time_ms=8.724 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2383gitlab # [ 411.137625] gitaly[920]: time="2026-11-02T13:16:59.735Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FVWADWFRX2Z7648H01PH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:59.723" pid=920 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc2384gitlab # [ 411.148051] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42d099c3c744d3686b401b08","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42d099c3c744d3686b401b08: done: 0.02019 sec","job_status":"done","queue_duration_s":2.7353,"scheduling_latency_s":2.735324,"redis_calls":13,"redis_duration_s":0.002277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02019,"completed_at":"2026-11-02T13:16:58.205Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2385gitlab # [ 411.188125] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61d4b391e888a7783d9d235a","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61d4b391e888a7783d9d235a: start","job_status":"start","queue_duration_s":2.75703,"scheduling_latency_s":2.757055}2386gitlab # [ 411.200727] gitaly[920]: time="2026-11-02T13:16:59.736Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M6E6FVWADWFRX2Z7648H01PH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-02T13:16:59.723" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=920 remote_ip="2001:db8:1::1" span.kind=server system=grpc2387gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/5bcc93bc627dd5c1ed48f56f948c7335c5310516/statuses, in 0.16 seconds)2388(finished: subtest: eval and build post commit statuses, in 11.83 seconds)2389(finished: run the VM test script, in 478.36 seconds)2390gitlab # [ 411.214255] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61d4b391e888a7783d9d235a","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61d4b391e888a7783d9d235a: done: 0.028266 sec","job_status":"done","queue_duration_s":2.75703,"scheduling_latency_s":2.757055,"redis_calls":13,"redis_duration_s":0.0032719999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00104,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028266,"completed_at":"2026-11-02T13:16:58.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2391test script finished in 478.39s2392cleanup2393kill QemuMachine (pid 46)2394gitlab # [ 411.254886] sidekiq-cluster[3287]: {"severity":"INFO","time":"2026-11-02T13:16:58.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b79e4be05db0b15d5207c8f6","created_at":"2026-11-02T13:16:55.323Z","trace_propagation_headers":{"sentry-trace":"7d789af699c447719f950f2fe635c5f3-5e155d7337a84337","baggage":"sentry-trace_id=7d789af699c447719f950f2fe635c5f3,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1672b6dbb0bb1b4559a2fcd066ad5e71","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-02T13:16:55.449Z","job_size_bytes":2,"pid":3287,"sidekiq_tid":"m8jj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b79e4be05db0b15d5207c8f6: start","job_status":"start","queue_duration_s":2.787586,"scheduling_latency_s":2.787619}2395gitlab # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2396(finished: cleanup, in 1.60 seconds)2397additionally exposed symbols:2398 gitlab,2399 vlan1,2400 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2401[]2402[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-02T13:15:18.102Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2403[]2404[]2405[]2406[]2407[]2408[]2409[]2410[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-02T13:16:47.715052Z', 'started_at': '2026-11-02T13:16:48.026765Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2411[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2412[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2413[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2414[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2415[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2416[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2417[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2418[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2419[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2420[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 10s (1 warning)', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': '2026-11-02T13:16:58.726Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-11-02T13:16:58.894Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2421[{'id': 5, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 10s (1 warning)', 'created_at': '2026-11-02T13:16:47.989Z', 'started_at': None, 'finished_at': '2026-11-02T13:16:58.726Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '5bcc93bc627dd5c1ed48f56f948c7335c5310516', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-11-02T13:16:58.894Z', 'started_at': None, 'finished_at': '2026-11-02T13:16:59.096Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]