vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #281
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14worker1: QEMU running (pid 50)15worker1 # Formatting '/build/vm-state-worker1/tmp.OEkcSPnhOU', fmt=raw size=107374182416lb # Disk image does not exist, creating the virtualisation disk image...17lb: QEMU running (pid 49)18worker2 # Disk image does not exist, creating the virtualisation disk image...19worker2: QEMU running (pid 52)20lb # Formatting '/build/vm-state-lb/tmp.sK0bVOoGxk', fmt=raw size=107374182421worker1 # mke2fs 1.47.4 (6-Mar-2025)22client: QEMU running (pid 51)23lb # mke2fs 1.47.4 (6-Mar-2025)24worker1 # Discarding device blocks: 0/262144 done25worker2 # Formatting '/build/vm-state-worker2/tmp.LdlDBJgtSs', fmt=raw size=107374182426client # Formatting '/build/vm-state-client/tmp.svgAz9zpVL', fmt=raw size=107374182427lb # Discarding device blocks: 0/262144 done28worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes29(finished: start all VMs, in 0.48 seconds)30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Filesystem UUID: d68d2f67-b566-43cd-a5a9-f5e3e0f2d05d32client: waiting for unit network-addresses-eth1.service33lb # Filesystem UUID: f5afaaae-8b76-4c33-87b6-c37da861016334worker1 # Superblock backups stored on blocks:35client: waiting for the VM to finish booting36lb # Superblock backups stored on blocks:37worker1 # 32768, 98304, 163840, 22937638client # mke2fs 1.47.4 (6-Mar-2025)39worker2 # mke2fs 1.47.4 (6-Mar-2025)40lb # 32768, 98304, 163840, 22937641worker1 # 42client # Discarding device blocks: 0/262144 done43worker2 # Discarding device blocks: 0/262144 done44lb # 45worker1 # Allocating group tables: 0/8 done46client # Creating filesystem with 262144 4k blocks and 65536 inodes47worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes48lb # Allocating group tables: 0/8 done49worker1 # Writing inode tables: 0/8 done50client # Filesystem UUID: 245aedea-c598-4d60-a5fa-ba226cc559af51worker2 # Filesystem UUID: 4ff74ba1-1420-4c2a-987c-fa456739f82a52lb # Writing inode tables: 0/8 done53worker1 # Creating journal (8192 blocks): done54client # Superblock backups stored on blocks:55worker2 # Superblock backups stored on blocks:56lb # Creating journal (8192 blocks): done57worker1 # Writing superblocks and filesystem accounting information: 0/8 done58client # 32768, 98304, 163840, 22937659worker2 # 32768, 98304, 163840, 22937660lb # Writing superblocks and filesystem accounting information: 0/8 done61worker1 # 62client # 63worker2 # 64lb # 65worker1 # Virtualisation disk image created.66client # Allocating group tables: 0/8 done67worker2 # Allocating group tables: 0/8 done68client # Writing inode tables: 0/8 done69worker2 # Writing inode tables: 0/8 done70client # Creating journal (8192 blocks): done71worker2 # Creating journal (8192 blocks): done72client # Writing superblocks and filesystem accounting information: 0/8 done73worker2 # Writing superblocks and filesystem accounting information: 0/8 done74client # 75worker2 # 76client # Virtualisation disk image created.77worker2 # Virtualisation disk image created.78lb # Virtualisation disk image created.79client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681client # [ 0.000000] KASLR enabled82client # [ 0.000000] random: crng init done83client # [ 0.000000] Machine model: linux,dummy-virt84client # [ 0.000000] efi: UEFI not found.85client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88client # [ 0.000000] Zone ranges:89client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90client # [ 0.000000] DMA32 empty91client # [ 0.000000] Normal empty92client # [ 0.000000] Device empty93client # [ 0.000000] Movable zone start for each node94client # [ 0.000000] Early memory node ranges95client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098client # [ 0.000000] psci: probing for conduit method from DT.99client # [ 0.000000] psci: PSCIv1.3 detected in firmware.100client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101client # [ 0.000000] psci: Trusted OS migration not required102client # [ 0.000000] psci: SMC Calling Convention v1.1103client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105client # [ 0.000000] Detected PIPT I-cache on CPU0106client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107client # [ 0.000000] CPU features: detected: GICv3 CPU interface108client # [ 0.000000] CPU features: detected: Spectre-v4109client # [ 0.000000] CPU features: detected: Spectre-BHB110client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112client # [ 0.000000] alternatives: applying boot alternatives113worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]114client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test/init regInfo=/nix/store/kw37kf8y651cwikgpy10xp7y2ldijiqs-closure-info/registration console=ttyAMA0,115200n8 console=tty0115worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026116worker1 # [ 0.000000] KASLR enabled117worker1 # [ 0.000000] random: crng init done118worker1 # [ 0.000000] Machine model: linux,dummy-virt119client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kw37kf8y651cwikgpy10xp7y2ldijiqs-closure-info/registration", will be passed to user space.120worker1 # [ 0.000000] efi: UEFI not found.121client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes122worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT123client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)124worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]125worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]126client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)127worker1 # [ 0.000000] Zone ranges:128client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB129worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]130client # [ 0.000000] software IO TLB: area num 1.131worker1 # [ 0.000000] DMA32 empty132worker1 # [ 0.000000] Normal empty133worker1 # [ 0.000000] Device empty134client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB135worker1 # [ 0.000000] Movable zone start for each node136worker1 # [ 0.000000] Early memory node ranges137client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)138client # [ 0.000000] Fallback order for Node 0: 0139worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]140client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216141worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] Policy zone: DMA143worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000144client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off145worker1 # [ 0.000000] psci: probing for conduit method from DT.146client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1147worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.148client # [ 0.000000] allocated 3145728 bytes of page_ext149worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs150client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages151worker1 # [ 0.000000] psci: Trusted OS migration not required152worker1 # [ 0.000000] psci: SMC Calling Convention v1.1153client # [ 0.000000] ftrace: allocated 294 pages with 4 groups154client # [ 0.000000] rcu: Hierarchical RCU implementation.155worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156client # [ 0.000000] rcu: RCU event tracing is enabled.157worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296158client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.159worker1 # [ 0.000000] Detected PIPT I-cache on CPU0160client # [ 0.000000] Trampoline variant of Tasks RCU enabled.161client # [ 0.000000] Rude variant of Tasks RCU enabled.162worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)163client # [ 0.000000] Tracing variant of Tasks RCU enabled.164worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface165worker1 # [ 0.000000] CPU features: detected: Spectre-v4166client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.167worker1 # [ 0.000000] CPU features: detected: Spectre-BHB168client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1169worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38170client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.171worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23172worker1 # [ 0.000000] alternatives: applying boot alternatives173client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.174client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.175client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0176client # [ 0.000000] GICv3: 256 SPIs implemented177client # [ 0.000000] GICv3: 0 Extended SPIs implemented178client # [ 0.000000] Root IRQ handler: gic_handle_irq179worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gy3k7x527c38djcmpycpycfhnskkzynp-nixos-system-worker1-test/init regInfo=/nix/store/b4g7ngbiscxlrax8kca3f93vbr2k88bd-closure-info/registration console=ttyAMA0,115200n8 console=tty0180client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI181client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0182worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/b4g7ngbiscxlrax8kca3f93vbr2k88bd-closure-info/registration", will be passed to user space.183client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000184client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]185worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes186worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)187client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)188worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)189client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)190worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB191client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000192worker1 # [ 0.000000] software IO TLB: area num 1.193client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000194worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB195client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.196worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)197worker1 # [ 0.000000] Fallback order for Node 0: 0198client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns199worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216200client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).201worker1 # [ 0.000000] Policy zone: DMA202worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off203client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns204worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1205worker1 # [ 0.000000] allocated 3145728 bytes of page_ext206client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns207client # [ 0.000035] arm-pv: using stolen time PV208worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages209lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]210worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups211client # [ 0.000652] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)212worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.213client # [ 0.000827] Console: colour dummy device 80x25214worker1 # [ 0.000000] rcu: RCU event tracing is enabled.215lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026216lb # [ 0.000000] KASLR enabled217client # [ 0.000835] printk: legacy console [tty0] enabled218lb # [ 0.000000] random: crng init done219worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.220lb # [ 0.000000] Machine model: linux,dummy-virt221worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.222lb # [ 0.000000] efi: UEFI not found.223client # [ 0.001035] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)224worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.225client # [ 0.001042] pid_max: default: 32768 minimum: 301226lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT227worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.228client # [ 0.001120] LSM: initializing lsm=capability,landlock,yama,bpf,ima229lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]230worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.231client # [ 0.001343] landlock: Up and running.232client # [ 0.001345] Yama: becoming mindful.233lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]234worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1235lb # [ 0.000000] Zone ranges:236client # [ 0.002005] LSM support for eBPF active237lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]238client # [ 0.002157] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)239worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.240lb # [ 0.000000] DMA32 empty241lb # [ 0.000000] Normal empty242client # [ 0.002191] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)243lb # [ 0.000000] Device empty244worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.245lb # [ 0.000000] Movable zone start for each node246client # [ 0.003364] cacheinfo: Unable to detect cache hierarchy for CPU 0247lb # [ 0.000000] Early memory node ranges248worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.249client # [ 0.004129] rcu: Hierarchical SRCU implementation.250lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]251worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0252client # [ 0.004133] rcu: Max phase no-delay instances is 1000.253worker1 # [ 0.000000] GICv3: 256 SPIs implemented254client # [ 0.005330] fsl-mc MSI: its@8080000 domain created255lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]256worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented257client # [ 0.005417] EFI services will not be available.258lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000259worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq260client # [ 0.005526] smp: Bringing up secondary CPUs ...261lb # [ 0.000000] psci: probing for conduit method from DT.262worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI263client # [ 0.005533] smp: Brought up 1 node, 1 CPU264lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.265worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0266client # [ 0.005536] SMP: Total of 1 processors activated.267worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]268lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs269client # [ 0.005541] CPU: All CPU(s) started at EL1270worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000271lb # [ 0.000000] psci: Trusted OS migration not required272client # [ 0.005551] CPU features: detected: Branch Target Identification273worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]274lb # [ 0.000000] psci: SMC Calling Convention v1.1275worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026276client # [ 0.005555] CPU features: detected: ARMv8.4 Translation Table Level277worker2 # [ 0.000000] KASLR enabled278worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)279worker2 # [ 0.000000] random: crng init done280lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)281worker2 # [ 0.000000] Machine model: linux,dummy-virt282client # [ 0.005558] CPU features: detected: Instruction cache invalidation not required for I/D coherence283lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296284worker2 # [ 0.000000] efi: UEFI not found.285worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)286lb # [ 0.000000] Detected PIPT I-cache on CPU0287client # [ 0.005562] CPU features: detected: Data cache clean to the PoU not required for I/D coherence288worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000289worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT290client # [ 0.005566] CPU features: detected: Common not Private translations291lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)292worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]293worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000294client # [ 0.005569] CPU features: detected: CRC32 instructions295lb # [ 0.000000] CPU features: detected: GICv3 CPU interface296worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]297lb # [ 0.000000] CPU features: detected: Spectre-v4298worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.299worker2 # [ 0.000000] Zone ranges:300client # [ 0.005571] CPU features: detected: Data cache clean to Point of Deep Persistence301lb # [ 0.000000] CPU features: detected: Spectre-BHB302worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]303client # [ 0.005575] CPU features: detected: Data cache clean to Point of Persistence304worker2 # [ 0.000000] DMA32 empty305lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38306worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns307worker2 # [ 0.000000] Normal empty308worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).309worker2 # [ 0.000000] Device empty310lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23311client # [ 0.005578] CPU features: detected: Data independent timing control (DIT)312worker2 # [ 0.000000] Movable zone start for each node313client # [ 0.005581] CPU features: detected: E0PD314lb # [ 0.000000] alternatives: applying boot alternatives315worker2 # [ 0.000000] Early memory node ranges316client # [ 0.005584] CPU features: detected: Enhanced Counter Virtualization317worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns318worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]319client # [ 0.005587] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)320worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns321worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]322worker1 # [ 0.000033] arm-pv: using stolen time PV323client # [ 0.005590] CPU features: detected: Enhanced Virtualization Traps324worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000325client # [ 0.005593] CPU features: detected: Fine Grained Traps326worker2 # [ 0.000000] psci: probing for conduit method from DT.327worker1 # [ 0.000525] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)328worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.329client # [ 0.005597] CPU features: detected: Generic authentication (architected QARMA5 algorithm)330worker1 # [ 0.000706] Console: colour dummy device 80x25331lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1y76sp58sczakc84qnb2p7nq6lq4cpg2-nixos-system-lb-test/init regInfo=/nix/store/yf668d8xpf1l4919kivs4ak64gjacscd-closure-info/registration console=ttyAMA0,115200n8 console=tty0332worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs333worker1 # [ 0.000713] printk: legacy console [tty0] enabled334client # [ 0.005601] CPU features: detected: RCpc load-acquire (LDAPR)335worker2 # [ 0.000000] psci: Trusted OS migration not required336client # [ 0.005604] CPU features: detected: LSE atomic instructions337worker2 # [ 0.000000] psci: SMC Calling Convention v1.1338worker1 # [ 0.000914] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)339lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/yf668d8xpf1l4919kivs4ak64gjacscd-closure-info/registration", will be passed to user space.340client # [ 0.005607] CPU features: detected: Privileged Access Never341worker1 # [ 0.000920] pid_max: default: 32768 minimum: 301342client # [ 0.005609] CPU features: detected: PMUv3343worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)344lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes345client # [ 0.005612] CPU features: detected: RAS Extension Support346worker1 # [ 0.000998] LSM: initializing lsm=capability,landlock,yama,bpf,ima347worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296348worker1 # [ 0.001151] landlock: Up and running.349lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)350client # [ 0.005614] CPU features: detected: RASv1p1 Extension Support351worker2 # [ 0.000000] Detected PIPT I-cache on CPU0352worker1 # [ 0.001154] Yama: becoming mindful.353client # [ 0.005617] CPU features: detected: Random Number Generator354worker1 # [ 0.001747] LSM support for eBPF active355lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)356worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)357client # [ 0.005620] CPU features: detected: Speculation barrier (SB)358worker1 # [ 0.001895] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)359worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface360client # [ 0.005622] CPU features: detected: Stage-2 Force Write-Back361worker2 # [ 0.000000] CPU features: detected: Spectre-v4362lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB363worker1 # [ 0.001926] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)364client # [ 0.005625] CPU features: detected: TLB range maintenance instructions365worker2 # [ 0.000000] CPU features: detected: Spectre-BHB366lb # [ 0.000000] software IO TLB: area num 1.367worker1 # [ 0.003040] cacheinfo: Unable to detect cache hierarchy for CPU 0368worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38369client # [ 0.005629] CPU features: detected: Speculative Store Bypassing Safe (SSBS)370lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB371worker1 # [ 0.003753] rcu: Hierarchical SRCU implementation.372worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23373client # [ 0.005670] alternatives: applying system-wide alternatives374worker1 # [ 0.003757] rcu: Max phase no-delay instances is 1000.375lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)376worker2 # [ 0.000000] alternatives: applying boot alternatives377worker1 # [ 0.004919] fsl-mc MSI: its@8080000 domain created378client # [ 0.008568] CPU features: detected: BBM Level 2 without TLB conflict abort379lb # [ 0.000000] Fallback order for Node 0: 0380worker1 # [ 0.005010] EFI services will not be available.381lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216382worker1 # [ 0.005074] smp: Bringing up secondary CPUs ...383lb # [ 0.000000] Policy zone: DMA384worker1 # [ 0.005082] smp: Brought up 1 node, 1 CPU385client # [ 0.008711] Memory: 1406500K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)386client # [ 0.009112] devtmpfs: initialized387lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off388worker1 # [ 0.005085] SMP: Total of 1 processors activated.389worker1 # [ 0.005091] CPU: All CPU(s) started at EL1390lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1391client # [ 0.010961] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)392worker1 # [ 0.005102] CPU features: detected: Branch Target Identification393lb # [ 0.000000] allocated 3145728 bytes of page_ext394worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jzdfkj5rn54mxddyl2kfg84vs1zgm4ba-nixos-system-worker2-test/init regInfo=/nix/store/a861za859qjzvnmb0mjbxjbnm95qmjfq-closure-info/registration console=ttyAMA0,115200n8 console=tty0395client # [ 0.010983] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).396lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages397worker1 # [ 0.005107] CPU features: detected: ARMv8.4 Translation Table Level398lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups399client # [ 0.011163] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL400worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/a861za859qjzvnmb0mjbxjbnm95qmjfq-closure-info/registration", will be passed to user space.401worker1 # [ 0.005110] CPU features: detected: Instruction cache invalidation not required for I/D coherence402lb # [ 0.000000] rcu: Hierarchical RCU implementation.403client # [ 0.011168] 0 pages in range for non-PLT usage404lb # [ 0.000000] rcu: RCU event tracing is enabled.405client # [ 0.011169] 508288 pages in range for PLT usage406worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes407worker1 # [ 0.005114] CPU features: detected: Data cache clean to the PoU not required for I/D coherence408client # [ 0.011279] pinctrl core: initialized pinctrl subsystem409lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.410client # [ 0.012059] DMI not present or invalid.411worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)412worker1 # [ 0.005118] CPU features: detected: Common not Private translations413lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.414worker1 # [ 0.005121] CPU features: detected: CRC32 instructions415client # [ 0.015702] NET: Registered PF_NETLINK/PF_ROUTE protocol family416lb # [ 0.000000] Rude variant of Tasks RCU enabled.417worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)418lb # [ 0.000000] Tracing variant of Tasks RCU enabled.419client # [ 0.018079] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations420worker1 # [ 0.005124] CPU features: detected: Data cache clean to Point of Deep Persistence421worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB422worker2 # [ 0.000000] software IO TLB: area num 1.423lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.424client # [ 0.018273] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations425worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB426worker1 # [ 0.005128] CPU features: detected: Data cache clean to Point of Persistence427lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1428client # [ 0.018473] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations429worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)430worker1 # [ 0.005131] CPU features: detected: Data independent timing control (DIT)431client # [ 0.018495] audit: initializing netlink subsys (disabled)432lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.433worker2 # [ 0.000000] Fallback order for Node 0: 0434worker1 # [ 0.005134] CPU features: detected: E0PD435client # [ 0.019074] thermal_sys: Registered thermal governor 'fair_share'436worker1 # [ 0.005137] CPU features: detected: Enhanced Counter Virtualization437worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216438lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.439worker2 # [ 0.000000] Policy zone: DMA440client # [ 0.019076] thermal_sys: Registered thermal governor 'bang_bang'441worker1 # [ 0.005139] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)442client # [ 0.019080] thermal_sys: Registered thermal governor 'step_wise'443worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off444lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.445worker1 # [ 0.005143] CPU features: detected: Enhanced Virtualization Traps446client # [ 0.019085] thermal_sys: Registered thermal governor 'user_space'447lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0448worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1449worker1 # [ 0.005146] CPU features: detected: Fine Grained Traps450lb # [ 0.000000] GICv3: 256 SPIs implemented451worker2 # [ 0.000000] allocated 3145728 bytes of page_ext452client # [ 0.019087] thermal_sys: Registered thermal governor 'power_allocator'453lb # [ 0.000000] GICv3: 0 Extended SPIs implemented454worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages455worker1 # [ 0.005150] CPU features: detected: Generic authentication (architected QARMA5 algorithm)456lb # [ 0.000000] Root IRQ handler: gic_handle_irq457client # [ 0.019111] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1458worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups459worker1 # [ 0.005155] CPU features: detected: RCpc load-acquire (LDAPR)460lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI461client # [ 0.019119] cpuidle: using governor ladder462worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.463client # [ 0.019125] cpuidle: using governor menu464worker1 # [ 0.005158] CPU features: detected: LSE atomic instructions465lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0466worker2 # [ 0.000000] rcu: RCU event tracing is enabled.467worker1 # [ 0.005161] CPU features: detected: Privileged Access Never468client # [ 0.019314] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.469lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000470worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.471worker1 # [ 0.005164] CPU features: detected: PMUv3472lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]473client # [ 0.019327] ASID allocator initialised with 65536 entries474worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.475worker1 # [ 0.005166] CPU features: detected: RAS Extension Support476client # [ 0.020591] Serial: AMBA PL011 UART driver477worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.478lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)479worker1 # [ 0.005169] CPU features: detected: RASv1p1 Extension Support480worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.481client # [ 0.025858] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1482worker1 # [ 0.005172] CPU features: detected: Random Number Generator483client # [ 0.026011] printk: console [ttyAMA0] enabled484worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.485lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)486worker1 # [ 0.005174] CPU features: detected: Speculation barrier (SB)487lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000488worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1489worker1 # [ 0.005177] CPU features: detected: Stage-2 Force Write-Back490lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000491worker1 # [ 0.005180] CPU features: detected: TLB range maintenance instructions492worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.493lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.494worker1 # [ 0.005185] CPU features: detected: Speculative Store Bypassing Safe (SSBS)495worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.496worker1 # [ 0.005224] alternatives: applying system-wide alternatives497lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns498worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.499worker1 # [ 0.008210] CPU features: detected: BBM Level 2 without TLB conflict abort500lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).501worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0502worker2 # [ 0.000000] GICv3: 256 SPIs implemented503worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented504lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns505worker1 # [ 0.008418] Memory: 1406520K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)506worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq507worker1 # [ 0.008800] devtmpfs: initialized508lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns509worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI510client # [ 0.145137] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages511worker1 # [ 0.010594] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)512lb # [ 0.000030] arm-pv: using stolen time PV513worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0514client # [ 0.145153] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page515worker1 # [ 0.010617] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).516worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000517lb # [ 0.000445] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)518client # [ 0.145158] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages519worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]520worker1 # [ 0.010833] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL521lb # [ 0.000613] Console: colour dummy device 80x25522client # [ 0.145162] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page523worker1 # [ 0.010837] 0 pages in range for non-PLT usage524lb # [ 0.000620] printk: legacy console [tty0] enabled525worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)526client # [ 0.145167] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages527worker1 # [ 0.010838] 508288 pages in range for PLT usage528worker1 # [ 0.010946] pinctrl core: initialized pinctrl subsystem529lb # [ 0.000814] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)530worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)531client # [ 0.145171] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page532worker1 # [ 0.011724] DMI not present or invalid.533lb # [ 0.000820] pid_max: default: 32768 minimum: 301534client # [ 0.145175] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages535worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000536worker1 # [ 0.015194] NET: Registered PF_NETLINK/PF_ROUTE protocol family537lb # [ 0.000887] LSM: initializing lsm=capability,landlock,yama,bpf,ima538client # [ 0.145180] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page539worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000540lb # [ 0.001026] landlock: Up and running.541worker1 # [ 0.017403] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations542lb # [ 0.001029] Yama: becoming mindful.543worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.544lb # [ 0.001527] LSM support for eBPF active545worker1 # [ 0.017597] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations546lb # [ 0.001670] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)547worker1 # [ 0.017798] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations548worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns549client # [ 0.152304] fbcon: Taking over console550client # [ 0.152318] ACPI: Interpreter disabled.551worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).552lb # [ 0.001702] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)553worker1 # [ 0.017818] audit: initializing netlink subsys (disabled)554worker1 # [ 0.018425] thermal_sys: Registered thermal governor 'fair_share'555lb # [ 0.002793] cacheinfo: Unable to detect cache hierarchy for CPU 0556worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns557lb # [ 0.003530] rcu: Hierarchical SRCU implementation.558worker1 # [ 0.018427] thermal_sys: Registered thermal governor 'bang_bang'559lb # [ 0.003534] rcu: Max phase no-delay instances is 1000.560worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns561worker1 # [ 0.018431] thermal_sys: Registered thermal governor 'step_wise'562lb # [ 0.004661] fsl-mc MSI: its@8080000 domain created563worker2 # [ 0.000030] arm-pv: using stolen time PV564worker1 # [ 0.018436] thermal_sys: Registered thermal governor 'user_space'565lb # [ 0.004757] EFI services will not be available.566worker1 # [ 0.018439] thermal_sys: Registered thermal governor 'power_allocator'567lb # [ 0.004819] smp: Bringing up secondary CPUs ...568worker2 # [ 0.000386] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)569lb # [ 0.004827] smp: Brought up 1 node, 1 CPU570worker2 # [ 0.000535] Console: colour dummy device 80x25571worker1 # [ 0.018464] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1572lb # [ 0.004830] SMP: Total of 1 processors activated.573worker2 # [ 0.000543] printk: legacy console [tty0] enabled574worker1 # [ 0.018472] cpuidle: using governor ladder575lb # [ 0.004835] CPU: All CPU(s) started at EL1576worker1 # [ 0.018478] cpuidle: using governor menu577lb # [ 0.004845] CPU features: detected: Branch Target Identification578worker2 # [ 0.000726] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)579worker1 # [ 0.018687] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.580lb # [ 0.004850] CPU features: detected: ARMv8.4 Translation Table Level581worker2 # [ 0.000733] pid_max: default: 32768 minimum: 301582worker1 # [ 0.018701] ASID allocator initialised with 65536 entries583worker2 # [ 0.000799] LSM: initializing lsm=capability,landlock,yama,bpf,ima584worker1 # [ 0.019948] Serial: AMBA PL011 UART driver585lb # [ 0.004853] CPU features: detected: Instruction cache invalidation not required for I/D coherence586worker2 # [ 0.000937] landlock: Up and running.587worker2 # [ 0.000939] Yama: becoming mindful.588client # [ 0.161317] iommu: Default domain type: Translated589worker1 # [ 0.025270] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1590worker2 # [ 0.001392] LSM support for eBPF active591lb # [ 0.004857] CPU features: detected: Data cache clean to the PoU not required for I/D coherence592client # [ 0.161327] iommu: DMA domain TLB invalidation policy: strict mode593worker1 # [ 0.025417] printk: console [ttyAMA0] enabled594lb # [ 0.004861] CPU features: detected: Common not Private translations595worker2 # [ 0.001529] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)596client # [ 0.161676] SCSI subsystem initialized597lb # [ 0.004864] CPU features: detected: CRC32 instructions598worker2 # [ 0.001561] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)599client # [ 0.163585] usbcore: registered new interface driver usbfs600lb # [ 0.004867] CPU features: detected: Data cache clean to Point of Deep Persistence601worker2 # [ 0.002665] cacheinfo: Unable to detect cache hierarchy for CPU 0602client # [ 0.163617] usbcore: registered new interface driver hub603worker2 # [ 0.003365] rcu: Hierarchical SRCU implementation.604lb # [ 0.004870] CPU features: detected: Data cache clean to Point of Persistence605client # [ 0.163636] usbcore: registered new device driver usb606worker2 # [ 0.003370] rcu: Max phase no-delay instances is 1000.607client # [ 0.163910] pps_core: LinuxPPS API ver. 1 registered608lb # [ 0.004874] CPU features: detected: Data independent timing control (DIT)609worker2 # [ 0.004537] fsl-mc MSI: its@8080000 domain created610lb # [ 0.004877] CPU features: detected: E0PD611worker2 # [ 0.004628] EFI services will not be available.612client # [ 0.163916] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>613lb # [ 0.004879] CPU features: detected: Enhanced Counter Virtualization614worker2 # [ 0.004690] smp: Bringing up secondary CPUs ...615client # [ 0.163926] PTP clock support registered616worker2 # [ 0.004697] smp: Brought up 1 node, 1 CPU617client # [ 0.163977] EDAC MC: Ver: 3.0.0618lb # [ 0.004882] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)619worker2 # [ 0.004701] SMP: Total of 1 processors activated.620client # [ 0.168445] scmi_core: SCMI protocol bus registered621lb # [ 0.004885] CPU features: detected: Enhanced Virtualization Traps622worker2 # [ 0.004706] CPU: All CPU(s) started at EL1623lb # [ 0.004888] CPU features: detected: Fine Grained Traps624worker2 # [ 0.004717] CPU features: detected: Branch Target Identification625client # [ 0.169401] FPGA manager framework626worker2 # [ 0.004722] CPU features: detected: ARMv8.4 Translation Table Level627client # [ 0.170400] vgaarb: loaded628lb # [ 0.004892] CPU features: detected: Generic authentication (architected QARMA5 algorithm)629lb # [ 0.004897] CPU features: detected: RCpc load-acquire (LDAPR)630worker2 # [ 0.004725] CPU features: detected: Instruction cache invalidation not required for I/D coherence631client # [ 0.171010] clocksource: Switched to clocksource arch_sys_counter632lb # [ 0.004900] CPU features: detected: LSE atomic instructions633worker1 # [ 0.147014] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages634lb # [ 0.004903] CPU features: detected: Privileged Access Never635worker2 # [ 0.004728] CPU features: detected: Data cache clean to the PoU not required for I/D coherence636worker1 # [ 0.147033] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page637lb # [ 0.004905] CPU features: detected: PMUv3638worker2 # [ 0.004732] CPU features: detected: Common not Private translations639worker1 # [ 0.147038] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages640lb # [ 0.004908] CPU features: detected: RAS Extension Support641worker2 # [ 0.004735] CPU features: detected: CRC32 instructions642lb # [ 0.004911] CPU features: detected: RASv1p1 Extension Support643worker1 # [ 0.147042] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page644worker2 # [ 0.004738] CPU features: detected: Data cache clean to Point of Deep Persistence645lb # [ 0.004914] CPU features: detected: Random Number Generator646worker1 # [ 0.147046] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages647worker2 # [ 0.004741] CPU features: detected: Data cache clean to Point of Persistence648lb # [ 0.004916] CPU features: detected: Speculation barrier (SB)649worker1 # [ 0.147050] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page650lb # [ 0.004920] CPU features: detected: Stage-2 Force Write-Back651worker2 # [ 0.004745] CPU features: detected: Data independent timing control (DIT)652worker1 # [ 0.147055] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages653worker2 # [ 0.004748] CPU features: detected: E0PD654lb # [ 0.004922] CPU features: detected: TLB range maintenance instructions655worker1 # [ 0.147059] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page656worker2 # [ 0.004750] CPU features: detected: Enhanced Counter Virtualization657lb # [ 0.004927] CPU features: detected: Speculative Store Bypassing Safe (SSBS)658client # [ 0.177499] VFS: Disk quotas dquot_6.6.0659worker2 # [ 0.004753] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)660lb # [ 0.004964] alternatives: applying system-wide alternatives661client # [ 0.177539] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)662worker2 # [ 0.004756] CPU features: detected: Enhanced Virtualization Traps663lb # [ 0.007881] CPU features: detected: BBM Level 2 without TLB conflict abort664client # [ 0.179219] netfs: FS-Cache loaded665worker1 # [ 0.154344] fbcon: Taking over console666worker2 # [ 0.004759] CPU features: detected: Fine Grained Traps667client # [ 0.179342] pnp: PnP ACPI: disabled668worker1 # [ 0.154359] ACPI: Interpreter disabled.669worker2 # [ 0.004763] CPU features: detected: Generic authentication (architected QARMA5 algorithm)670lb # [ 0.008072] Memory: 1406496K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)671worker1 # [ 0.156227] iommu: Default domain type: Translated672worker2 # [ 0.004767] CPU features: detected: RCpc load-acquire (LDAPR)673lb # [ 0.008439] devtmpfs: initialized674worker1 # [ 0.156236] iommu: DMA domain TLB invalidation policy: strict mode675worker2 # [ 0.004770] CPU features: detected: LSE atomic instructions676lb # [ 0.010155] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)677worker2 # [ 0.004773] CPU features: detected: Privileged Access Never678worker1 # [ 0.157904] SCSI subsystem initialized679worker2 # [ 0.004775] CPU features: detected: PMUv3680lb # [ 0.010176] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).681worker2 # [ 0.004778] CPU features: detected: RAS Extension Support682lb # [ 0.010357] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL683worker2 # [ 0.004781] CPU features: detected: RASv1p1 Extension Support684lb # [ 0.010361] 0 pages in range for non-PLT usage685worker2 # [ 0.004783] CPU features: detected: Random Number Generator686lb # [ 0.010362] 508288 pages in range for PLT usage687lb # [ 0.010461] pinctrl core: initialized pinctrl subsystem688worker2 # [ 0.004786] CPU features: detected: Speculation barrier (SB)689lb # [ 0.011238] DMI not present or invalid.690worker2 # [ 0.004789] CPU features: detected: Stage-2 Force Write-Back691lb # [ 0.014396] NET: Registered PF_NETLINK/PF_ROUTE protocol family692worker2 # [ 0.004792] CPU features: detected: TLB range maintenance instructions693client # [ 0.183184] NET: Registered PF_INET protocol family694lb # [ 0.016564] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations695worker2 # [ 0.004797] CPU features: detected: Speculative Store Bypassing Safe (SSBS)696client # [ 0.183474] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)697worker1 # [ 0.163256] usbcore: registered new interface driver usbfs698worker2 # [ 0.004833] alternatives: applying system-wide alternatives699lb # [ 0.016732] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations700worker1 # [ 0.163289] usbcore: registered new interface driver hub701worker2 # [ 0.007767] CPU features: detected: BBM Level 2 without TLB conflict abort702worker1 # [ 0.163306] usbcore: registered new device driver usb703lb # [ 0.016923] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations704worker1 # [ 0.163573] pps_core: LinuxPPS API ver. 1 registered705lb # [ 0.016944] audit: initializing netlink subsys (disabled)706worker2 # [ 0.007938] Memory: 1406512K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)707lb # [ 0.017522] thermal_sys: Registered thermal governor 'fair_share'708worker1 # [ 0.163579] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>709worker2 # [ 0.008342] devtmpfs: initialized710lb # [ 0.017524] thermal_sys: Registered thermal governor 'bang_bang'711worker1 # [ 0.163589] PTP clock support registered712worker1 # [ 0.163639] EDAC MC: Ver: 3.0.0713worker2 # [ 0.010055] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)714lb # [ 0.017527] thermal_sys: Registered thermal governor 'step_wise'715worker1 # [ 0.168102] scmi_core: SCMI protocol bus registered716lb # [ 0.017533] thermal_sys: Registered thermal governor 'user_space'717worker2 # [ 0.010076] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).718lb # [ 0.017535] thermal_sys: Registered thermal governor 'power_allocator'719worker2 # [ 0.010256] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL720worker1 # [ 0.169068] FPGA manager framework721worker1 # [ 0.170022] vgaarb: loaded722worker2 # [ 0.010261] 0 pages in range for non-PLT usage723lb # [ 0.017559] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1724worker2 # [ 0.010262] 508288 pages in range for PLT usage725lb # [ 0.017569] cpuidle: using governor ladder726worker1 # [ 0.170645] clocksource: Switched to clocksource arch_sys_counter727worker2 # [ 0.010365] pinctrl core: initialized pinctrl subsystem728worker2 # [ 0.011134] DMI not present or invalid.729worker2 # [ 0.014216] NET: Registered PF_NETLINK/PF_ROUTE protocol family730worker2 # [ 0.016387] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations731worker2 # [ 0.016563] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations732worker1 # [ 0.174465] VFS: Disk quotas dquot_6.6.0733worker2 # [ 0.016755] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations734worker1 # [ 0.174500] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)735worker2 # [ 0.016773] audit: initializing netlink subsys (disabled)736worker2 # [ 0.017325] thermal_sys: Registered thermal governor 'fair_share'737worker2 # [ 0.017327] thermal_sys: Registered thermal governor 'bang_bang'738worker2 # [ 0.017331] thermal_sys: Registered thermal governor 'step_wise'739worker1 # [ 0.178281] netfs: FS-Cache loaded740worker2 # [ 0.017336] thermal_sys: Registered thermal governor 'user_space'741worker1 # [ 0.178409] pnp: PnP ACPI: disabled742lb # [ 0.017574] cpuidle: using governor menu743worker2 # [ 0.017338] thermal_sys: Registered thermal governor 'power_allocator'744lb # [ 0.017762] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.745lb # [ 0.017777] ASID allocator initialised with 65536 entries746worker2 # [ 0.017362] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1747lb # [ 0.018951] Serial: AMBA PL011 UART driver748worker2 # [ 0.017372] cpuidle: using governor ladder749worker2 # [ 0.017378] cpuidle: using governor menu750lb # [ 0.024161] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1751worker2 # [ 0.017562] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.752lb # [ 0.024277] printk: console [ttyAMA0] enabled753worker1 # [ 0.182421] NET: Registered PF_INET protocol family754worker2 # [ 0.017577] ASID allocator initialised with 65536 entries755worker2 # [ 0.018775] Serial: AMBA PL011 UART driver756lb # [ 0.145365] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages757worker2 # [ 0.023821] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1758lb # [ 0.145385] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page759worker2 # [ 0.023974] printk: console [ttyAMA0] enabled760lb # [ 0.145390] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages761lb # [ 0.145394] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page762worker1 # [ 0.183296] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)763lb # [ 0.145399] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages764lb # [ 0.145403] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page765lb # [ 0.145407] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages766lb # [ 0.145411] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page767lb # [ 0.152891] fbcon: Taking over console768worker2 # [ 0.140828] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages769lb # [ 0.152907] ACPI: Interpreter disabled.770worker2 # [ 0.140846] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page771lb # [ 0.154841] iommu: Default domain type: Translated772worker2 # [ 0.140851] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages773lb # [ 0.154851] iommu: DMA domain TLB invalidation policy: strict mode774worker2 # [ 0.140855] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page775lb # [ 0.156538] SCSI subsystem initialized776worker2 # [ 0.140859] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages777client # [ 0.212550] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)778worker2 # [ 0.140863] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page779client # [ 0.212592] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)780worker2 # [ 0.140868] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages781client # [ 0.212619] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)782worker2 # [ 0.140872] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page783client # [ 0.212694] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)784client # [ 0.212837] TCP: Hash tables configured (established 16384 bind 16384)785client # [ 0.212916] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)786worker2 # [ 0.148251] fbcon: Taking over console787worker2 # [ 0.148266] ACPI: Interpreter disabled.788client # [ 0.212951] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)789lb # [ 0.161766] usbcore: registered new interface driver usbfs790client # [ 0.212986] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)791lb # [ 0.161799] usbcore: registered new interface driver hub792client # [ 0.213069] NET: Registered PF_UNIX/PF_LOCAL protocol family793lb # [ 0.161819] usbcore: registered new device driver usb794client # [ 0.213091] NET: Registered PF_XDP protocol family795lb # [ 0.162128] pps_core: LinuxPPS API ver. 1 registered796client # [ 0.213107] PCI: CLS 0 bytes, default 64797lb # [ 0.162134] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>798client # [ 0.213340] Trying to unpack rootfs image as initramfs...799lb # [ 0.162144] PTP clock support registered800lb # [ 0.162193] EDAC MC: Ver: 3.0.0801lb # [ 0.166848] scmi_core: SCMI protocol bus registered802lb # [ 0.167842] FPGA manager framework803client # [ 0.228624] kvm [1]: HYP mode not available804lb # [ 0.168868] vgaarb: loaded805lb # [ 0.169516] clocksource: Switched to clocksource arch_sys_counter806worker2 # [ 0.157256] iommu: Default domain type: Translated807worker2 # [ 0.157267] iommu: DMA domain TLB invalidation policy: strict mode808worker2 # [ 0.157622] SCSI subsystem initialized809worker2 # [ 0.159608] usbcore: registered new interface driver usbfs810worker2 # [ 0.159640] usbcore: registered new interface driver hub811lb # [ 0.173366] VFS: Disk quotas dquot_6.6.0812worker2 # [ 0.159664] usbcore: registered new device driver usb813lb # [ 0.173405] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)814worker2 # [ 0.159936] pps_core: LinuxPPS API ver. 1 registered815worker2 # [ 0.159942] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>816worker2 # [ 0.159952] PTP clock support registered817worker2 # [ 0.160005] EDAC MC: Ver: 3.0.0818lb # [ 0.177036] netfs: FS-Cache loaded819worker2 # [ 0.164660] scmi_core: SCMI protocol bus registered820lb # [ 0.177152] pnp: PnP ACPI: disabled821worker2 # [ 0.165622] FPGA manager framework822worker2 # [ 0.166587] vgaarb: loaded823worker2 # [ 0.167219] clocksource: Switched to clocksource arch_sys_counter824worker1 # [ 0.212801] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)825lb # [ 0.181043] NET: Registered PF_INET protocol family826worker1 # [ 0.212842] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)827lb # [ 0.181296] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)828worker1 # [ 0.212869] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)829worker1 # [ 0.212939] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)830worker1 # [ 0.213084] TCP: Hash tables configured (established 16384 bind 16384)831worker1 # [ 0.213162] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)832worker2 # [ 0.173333] VFS: Disk quotas dquot_6.6.0833worker1 # [ 0.213199] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)834worker2 # [ 0.173374] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)835worker1 # [ 0.213238] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)836worker2 # [ 0.175022] netfs: FS-Cache loaded837worker1 # [ 0.213331] NET: Registered PF_UNIX/PF_LOCAL protocol family838worker2 # [ 0.175148] pnp: PnP ACPI: disabled839worker1 # [ 0.213401] NET: Registered PF_XDP protocol family840worker1 # [ 0.213419] PCI: CLS 0 bytes, default 64841worker1 # [ 0.213646] Trying to unpack rootfs image as initramfs...842worker2 # [ 0.179170] NET: Registered PF_INET protocol family843worker2 # [ 0.180049] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)844worker1 # [ 0.228369] kvm [1]: HYP mode not available845lb # [ 0.210766] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846lb # [ 0.210817] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847lb # [ 0.210847] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848lb # [ 0.210921] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849lb # [ 0.211066] TCP: Hash tables configured (established 16384 bind 16384)850lb # [ 0.211166] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851lb # [ 0.211203] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)852lb # [ 0.211239] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)853lb # [ 0.211346] NET: Registered PF_UNIX/PF_LOCAL protocol family854lb # [ 0.211398] NET: Registered PF_XDP protocol family855lb # [ 0.211416] PCI: CLS 0 bytes, default 64856lb # [ 0.211671] Trying to unpack rootfs image as initramfs...857worker2 # [ 0.208602] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)858worker2 # [ 0.208650] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)859worker2 # [ 0.208678] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)860worker2 # [ 0.208751] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)861worker2 # [ 0.208895] TCP: Hash tables configured (established 16384 bind 16384)862worker2 # [ 0.208975] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)863worker2 # [ 0.209020] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)864lb # [ 0.227282] kvm [1]: HYP mode not available865worker2 # [ 0.209055] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866worker2 # [ 0.209149] NET: Registered PF_UNIX/PF_LOCAL protocol family867worker2 # [ 0.209200] NET: Registered PF_XDP protocol family868worker2 # [ 0.209219] PCI: CLS 0 bytes, default 64869worker2 # [ 0.209451] Trying to unpack rootfs image as initramfs...870worker2 # [ 0.224856] kvm [1]: HYP mode not available871client # [ 0.321494] Initialise system trusted keyrings872client # [ 0.322278] workingset: timestamp_bits=42 max_order=19 bucket_order=0873client # [ 0.323584] squashfs: version 4.0 (2009/01/31) Phillip Lougher874client # [ 0.324345] 9p: Installing v9fs 9p2000 file system support875worker1 # [ 0.319185] Initialise system trusted keyrings876client # [ 0.344249] Key type asymmetric registered877worker1 # [ 0.319938] workingset: timestamp_bits=42 max_order=19 bucket_order=0878client # [ 0.344274] Asymmetric key parser 'x509' registered879worker1 # [ 0.321204] squashfs: version 4.0 (2009/01/31) Phillip Lougher880client # [ 0.344339] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)881worker1 # [ 0.321997] 9p: Installing v9fs 9p2000 file system support882client # [ 0.351056] io scheduler mq-deadline registered883client # [ 0.351074] io scheduler kyber registered884client # [ 0.363162] pl061_gpio 9030000.pl061: PL061 GPIO chip registered885client # [ 0.364572] ledtrig-cpu: registered to indicate activity on CPUs886client # [ 0.364975] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:887client # [ 0.364991] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000888client # [ 0.365003] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000889worker1 # [ 0.345905] Key type asymmetric registered890worker1 # [ 0.345932] Asymmetric key parser 'x509' registered891client # [ 0.365011] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000892worker1 # [ 0.345998] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)893client # [ 0.365031] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits894client # [ 0.365055] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]895client # [ 0.365131] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00896client # [ 0.365140] pci_bus 0000:00: root bus resource [bus 00-ff]897client # [ 0.365147] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]898client # [ 0.365152] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]899client # [ 0.365156] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]900client # [ 0.365214] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint901client # [ 0.365642] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint902client # [ 0.365858] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]903client # [ 0.365875] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]904client # [ 0.365904] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]905client # [ 0.365920] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]906client # [ 0.366364] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint907worker1 # [ 0.354700] io scheduler mq-deadline registered908worker1 # [ 0.354722] io scheduler kyber registered909client # [ 0.366542] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]910client # [ 0.366558] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]911lb # [ 0.322107] Initialise system trusted keyrings912worker1 # [ 0.360090] pl061_gpio 9030000.pl061: PL061 GPIO chip registered913client # [ 0.366586] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]914lb # [ 0.322853] workingset: timestamp_bits=42 max_order=19 bucket_order=0915lb # [ 0.324136] squashfs: version 4.0 (2009/01/31) Phillip Lougher916worker1 # [ 0.361393] ledtrig-cpu: registered to indicate activity on CPUs917client # [ 0.385843] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint918worker1 # [ 0.361771] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:919client # [ 0.386029] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]920lb # [ 0.324895] 9p: Installing v9fs 9p2000 file system support921client # [ 0.386045] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]922worker1 # [ 0.361788] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000923client # [ 0.386074] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]924worker1 # [ 0.361806] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000925client # [ 0.386536] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint926worker2 # [ 0.315752] Initialise system trusted keyrings927client # [ 0.386715] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]928worker1 # [ 0.361815] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000929client # [ 0.386731] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]930worker2 # [ 0.316484] workingset: timestamp_bits=42 max_order=19 bucket_order=0931worker1 # [ 0.361840] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits932client # [ 0.386762] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]933worker2 # [ 0.317718] squashfs: version 4.0 (2009/01/31) Phillip Lougher934worker1 # [ 0.361860] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]935worker1 # [ 0.361937] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00936worker2 # [ 0.318466] 9p: Installing v9fs 9p2000 file system support937worker1 # [ 0.361946] pci_bus 0000:00: root bus resource [bus 00-ff]938worker1 # [ 0.361953] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]939worker1 # [ 0.361958] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]940worker1 # [ 0.361962] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]941worker1 # [ 0.362019] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint942worker1 # [ 0.362448] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint943worker1 # [ 0.362634] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]944client # [ 0.399274] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint945client # [ 0.399466] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]946client # [ 0.399483] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]947client # [ 0.399512] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]948client # [ 0.399962] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint949client # [ 0.400143] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]950client # [ 0.400159] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]951client # [ 0.400188] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]952client # [ 0.400630] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint953worker2 # [ 0.339093] Key type asymmetric registered954client # [ 0.400808] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]955worker2 # [ 0.339119] Asymmetric key parser 'x509' registered956worker1 # [ 0.382678] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]957client # [ 0.400824] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]958worker2 # [ 0.339181] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)959worker1 # [ 0.382739] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]960client # [ 0.400852] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]961lb # [ 0.353578] Key type asymmetric registered962worker1 # [ 0.382757] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]963client # [ 0.400868] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]964lb # [ 0.353608] Asymmetric key parser 'x509' registered965worker1 # [ 0.383272] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint966client # [ 0.401339] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint967lb # [ 0.353679] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)968worker1 # [ 0.383466] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]969client # [ 0.401522] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]970lb # [ 0.355754] io scheduler mq-deadline registered971lb # [ 0.355765] io scheduler kyber registered972worker1 # [ 0.383482] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]973client # [ 0.401552] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]974worker1 # [ 0.383511] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975client # [ 0.402003] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint976worker1 # [ 0.383970] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint977client # [ 0.402186] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]978worker1 # [ 0.384154] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]979client # [ 0.402214] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980worker1 # [ 0.384169] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]981worker2 # [ 0.347289] io scheduler mq-deadline registered982client # [ 0.402588] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint983worker2 # [ 0.347308] io scheduler kyber registered984worker1 # [ 0.384199] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]985client # [ 0.402765] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]986worker1 # [ 0.384649] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint987client # [ 0.403008] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint988worker1 # [ 0.384831] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]989client # [ 0.403292] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]990worker1 # [ 0.384847] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]991client # [ 0.403309] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]992worker1 # [ 0.384879] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993client # [ 0.403338] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.385373] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint995client # [ 0.403791] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint996worker1 # [ 0.385556] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]997client # [ 0.403969] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]998worker1 # [ 0.385572] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]999client # [ 0.403985] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1000lb # [ 0.365632] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1001worker1 # [ 0.385600] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1002client # [ 0.404013] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1003worker1 # [ 0.386049] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1004client # [ 0.404569] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1005worker1 # [ 0.386232] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1006client # [ 0.404581] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1007worker1 # [ 0.386247] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1008lb # [ 0.366973] ledtrig-cpu: registered to indicate activity on CPUs1009client # [ 0.404587] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1010worker1 # [ 0.386276] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011lb # [ 0.367414] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1012client # [ 0.404629] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1013worker1 # [ 0.386748] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1014worker1 # [ 0.386932] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1015lb # [ 0.367432] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001016client # [ 0.404675] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1017worker1 # [ 0.386948] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1018lb # [ 0.367444] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001019client # [ 0.404719] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1020worker1 # [ 0.386976] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1021client # [ 0.404765] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1022lb # [ 0.367453] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001023worker1 # [ 0.386992] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1024client # [ 0.404810] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1025worker1 # [ 0.387443] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1026lb # [ 0.367474] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1027worker1 # [ 0.387629] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1028client # [ 0.404855] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1029lb # [ 0.367497] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1030worker1 # [ 0.387659] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1031client # [ 0.404900] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1032lb # [ 0.367585] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001033worker1 # [ 0.388115] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1034lb # [ 0.367596] pci_bus 0000:00: root bus resource [bus 00-ff]1035client # [ 0.404946] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1036worker1 # [ 0.388303] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1037lb # [ 0.367602] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1038client # [ 0.404991] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1039worker1 # [ 0.388332] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1040lb # [ 0.367607] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1041client # [ 0.405050] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1042lb # [ 0.367612] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1043worker1 # [ 0.388717] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1044worker2 # [ 0.359368] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1045client # [ 0.405096] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1046worker1 # [ 0.388898] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1047worker2 # [ 0.359982] ledtrig-cpu: registered to indicate activity on CPUs1048lb # [ 0.367700] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1049client # [ 0.405116] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1050worker2 # [ 0.360349] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1051worker1 # [ 0.389141] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1052lb # [ 0.368135] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1053client # [ 0.405137] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1054worker1 # [ 0.389424] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1055lb # [ 0.368321] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1056client # [ 0.405158] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1057worker2 # [ 0.360373] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001058worker1 # [ 0.389441] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1059lb # [ 0.368338] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1060client # [ 0.405182] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1061worker2 # [ 0.360386] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001062worker1 # [ 0.389470] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1063lb # [ 0.368367] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064client # [ 0.405203] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1065lb # [ 0.368384] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1066worker2 # [ 0.360394] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001067worker1 # [ 0.389935] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1068client # [ 0.405224] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1069worker1 # [ 0.390118] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1070lb # [ 0.368839] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1071worker2 # [ 0.360413] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1072client # [ 0.405246] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1073worker1 # [ 0.390133] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1074lb # [ 0.369024] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1075client # [ 0.405267] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1076worker2 # [ 0.360435] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1077worker1 # [ 0.390162] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078lb # [ 0.369040] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1079client # [ 0.405288] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1080worker2 # [ 0.360511] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001081lb # [ 0.369070] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082client # [ 0.405310] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1083worker2 # [ 0.360520] pci_bus 0000:00: root bus resource [bus 00-ff]1084worker2 # [ 0.360527] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1085client # [ 0.405332] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1086client # [ 0.405353] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1087worker2 # [ 0.360532] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1088lb # [ 0.388074] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1089client # [ 0.405375] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1090worker2 # [ 0.360538] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1091lb # [ 0.388260] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1092client # [ 0.405395] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1093lb # [ 0.388276] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1094worker2 # [ 0.360613] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1095client # [ 0.405416] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1096lb # [ 0.388306] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1097worker2 # [ 0.361049] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098client # [ 0.405437] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1099worker2 # [ 0.361236] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1100client # [ 0.405458] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1101lb # [ 0.388773] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1102worker2 # [ 0.361252] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1103lb # [ 0.388957] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1104client # [ 0.405479] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1105lb # [ 0.388973] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1106client # [ 0.405501] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1107worker2 # [ 0.361282] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108client # [ 0.405522] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1109worker2 # [ 0.361298] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1110lb # [ 0.389005] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111client # [ 0.405548] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1112lb # [ 0.389466] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1113worker2 # [ 0.361755] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1114client # [ 0.405557] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1115worker2 # [ 0.361939] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1116worker1 # [ 0.430816] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1117client # [ 0.405562] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1118worker2 # [ 0.361955] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1119worker1 # [ 0.430837] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1120client # [ 0.406692] pci 0000:00:0a.0: enabling device (0000 -> 0002)1121worker2 # [ 0.361985] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122worker1 # [ 0.430842] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1123worker2 # [ 0.362430] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1124worker1 # [ 0.430892] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1125worker2 # [ 0.362610] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1126worker2 # [ 0.362626] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1127worker1 # [ 0.430940] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1128worker2 # [ 0.362669] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129worker1 # [ 0.430987] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1130worker2 # [ 0.363118] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1131worker1 # [ 0.431033] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1132worker2 # [ 0.363319] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1133worker1 # [ 0.431081] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1134worker2 # [ 0.363335] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1135worker1 # [ 0.431128] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1136worker2 # [ 0.363369] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137lb # [ 0.401681] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1138worker1 # [ 0.431176] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1139worker2 # [ 0.363828] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1140client # [ 0.467132] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 59028 usecs1141worker2 # [ 0.364010] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1142lb # [ 0.401709] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker1 # [ 0.431224] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1144worker2 # [ 0.364026] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1145lb # [ 0.401739] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146worker1 # [ 0.431271] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1147worker2 # [ 0.364056] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148lb # [ 0.402215] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1149worker1 # [ 0.431379] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1150lb # [ 0.402403] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1151worker2 # [ 0.364513] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1152worker1 # [ 0.431428] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1153lb # [ 0.402419] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1154worker2 # [ 0.364695] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1155worker2 # [ 0.364710] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1156worker1 # [ 0.431449] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1157lb # [ 0.402449] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158worker2 # [ 0.364740] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1159worker1 # [ 0.431471] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1160lb # [ 0.402905] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1161worker1 # [ 0.431493] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1162lb # [ 0.403090] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1163worker2 # [ 0.365185] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1164lb # [ 0.403106] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1165worker2 # [ 0.365370] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1166worker1 # [ 0.431518] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1167worker2 # [ 0.365385] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1168lb # [ 0.403144] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169worker1 # [ 0.431540] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1170worker2 # [ 0.365415] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171lb # [ 0.403161] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1172worker1 # [ 0.431562] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1173worker2 # [ 0.365431] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1174lb # [ 0.403651] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1175worker1 # [ 0.431584] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1176worker2 # [ 0.365909] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1177lb # [ 0.403840] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1178worker1 # [ 0.431606] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1179worker2 # [ 0.366096] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1180lb # [ 0.403870] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1181worker1 # [ 0.431628] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1182worker2 # [ 0.366126] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183worker1 # [ 0.431651] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1184lb # [ 0.404323] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1185worker2 # [ 0.366571] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1186worker1 # [ 0.431674] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1187lb # [ 0.404515] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1188worker2 # [ 0.366770] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1189worker1 # [ 0.431695] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1190lb # [ 0.404544] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1191worker1 # [ 0.431716] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1192worker2 # [ 0.366800] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1193lb # [ 0.404945] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1194worker1 # [ 0.431738] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1195lb # [ 0.405131] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1196worker2 # [ 0.367188] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1197worker1 # [ 0.431759] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1198lb # [ 0.405374] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1199worker1 # [ 0.431780] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1200worker1 # [ 0.431802] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1201worker1 # [ 0.431823] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1202worker1 # [ 0.431844] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1203worker1 # [ 0.431866] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1204worker1 # [ 0.431892] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1205lb # [ 0.422249] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1206worker1 # [ 0.431901] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1207lb # [ 0.422270] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1208worker1 # [ 0.431906] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1209lb # [ 0.422300] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1210worker1 # [ 0.433071] pci 0000:00:0a.0: enabling device (0000 -> 0002)1211lb # [ 0.422760] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1212lb # [ 0.422943] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1213client # [ 0.487570] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1214lb # [ 0.422958] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1215client # [ 0.489627] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1216lb # [ 0.422988] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1217lb # [ 0.423591] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1218worker2 # [ 0.415428] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1219lb # [ 0.423603] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1220worker2 # [ 0.415713] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1221lb # [ 0.423608] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1222worker2 # [ 0.415978] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1223lb # [ 0.423652] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1224worker2 # [ 0.415995] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1225lb # [ 0.423698] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1226worker2 # [ 0.416025] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1227lb # [ 0.423745] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1228worker2 # [ 0.416475] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1229worker2 # [ 0.416657] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1230lb # [ 0.423791] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1231worker2 # [ 0.416673] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1232lb # [ 0.423838] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1233worker2 # [ 0.416702] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1234lb # [ 0.423885] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1235worker2 # [ 0.417268] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1236lb # [ 0.423931] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1237worker2 # [ 0.417279] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1238lb # [ 0.423977] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1239client # [ 0.499647] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1240worker2 # [ 0.417285] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1241client # [ 0.501859] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1242lb # [ 0.424024] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1243worker2 # [ 0.417329] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1244lb # [ 0.424101] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1245worker2 # [ 0.417376] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1246lb # [ 0.424148] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1247worker2 # [ 0.417423] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1248lb # [ 0.424169] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1249worker2 # [ 0.417470] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1250lb # [ 0.424191] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1251worker2 # [ 0.417516] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1252lb # [ 0.424213] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1253lb # [ 0.424237] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1254worker2 # [ 0.417564] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1255lb # [ 0.424259] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1256worker2 # [ 0.417611] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1257lb # [ 0.424281] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1258worker2 # [ 0.417659] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1259lb # [ 0.424304] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1260worker2 # [ 0.417706] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1261lb # [ 0.424326] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1262worker2 # [ 0.417768] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1263lb # [ 0.424348] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1264worker2 # [ 0.417813] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1265lb # [ 0.424370] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1266worker2 # [ 0.417835] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1267lb # [ 0.424393] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1268worker1 # [ 0.489310] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1269client # [ 0.511581] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1270lb # [ 0.424415] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1271worker2 # [ 0.417856] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1272client # [ 0.514372] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1273lb # [ 0.424436] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1274worker2 # [ 0.417878] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1275lb # [ 0.424458] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1276worker2 # [ 0.417902] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1277lb # [ 0.424479] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1278worker2 # [ 0.417923] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1279lb # [ 0.424500] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1280worker2 # [ 0.417945] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1281lb # [ 0.424522] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1282worker2 # [ 0.417967] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1283lb # [ 0.424544] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1284worker2 # [ 0.417989] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1285lb # [ 0.424566] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1286worker2 # [ 0.418011] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1287lb # [ 0.424588] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1288lb # [ 0.424615] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1289worker2 # [ 0.418033] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1290worker1 # [ 0.495249] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1291lb # [ 0.424625] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1292worker2 # [ 0.418056] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1293lb # [ 0.424629] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1294worker2 # [ 0.418077] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1295worker1 # [ 0.497450] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1296worker2 # [ 0.418098] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1297worker2 # [ 0.418120] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1298worker2 # [ 0.418141] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1299worker2 # [ 0.418162] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1300client # [ 0.525603] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1301worker2 # [ 0.418183] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1302worker2 # [ 0.418205] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1303worker2 # [ 0.418226] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1304worker2 # [ 0.418248] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1305worker2 # [ 0.418274] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1306client # [ 0.528544] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1307worker2 # [ 0.418283] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1308client # [ 0.530644] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1309worker2 # [ 0.418288] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1310worker2 # [ 0.458224] pci 0000:00:0a.0: enabling device (0000 -> 0002)1311worker1 # [ 0.507804] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1312worker1 # [ 0.509759] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1313lb # [ 0.473899] pci 0000:00:0a.0: enabling device (0000 -> 0002)1314worker1 # [ 0.513351] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1315client # [ 0.540521] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1316client # [ 0.542817] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1317worker1 # [ 0.519905] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1318worker2 # [ 0.482235] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1319lb # [ 0.494874] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1320lb # [ 0.496965] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1321worker2 # [ 0.488402] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1322client # [ 0.559449] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1323worker1 # [ 0.534849] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1324client # [ 0.561200] msm_serial: driver initialized1325worker2 # [ 0.490450] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1326worker1 # [ 0.537026] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1327client # [ 0.561333] SuperH (H)SCI(F) driver initialized1328client # [ 0.561386] STM32 USART driver initialized1329worker1 # [ 0.538840] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1330worker1 # [ 0.541189] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1331lb # [ 0.508165] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1332worker2 # [ 0.500626] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333worker2 # [ 0.502502] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1334worker1 # [ 0.558809] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1335worker1 # [ 0.561545] msm_serial: driver initialized1336worker1 # [ 0.561694] SuperH (H)SCI(F) driver initialized1337worker1 # [ 0.561747] STM32 USART driver initialized1338lb # [ 0.518218] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1339lb # [ 0.521929] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1340lb # [ 0.523991] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1341worker2 # [ 0.513888] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1342client # [ 0.589733] loop: module loaded1343client # [ 0.589916] virtio_blk virtio5: 1/0/0 default/read/poll queues1344client # [ 0.590648] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1345worker2 # [ 0.517035] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1346worker2 # [ 0.519099] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1347client # [ 0.595587] megasas: 07.734.00.00-rc11348client # [ 0.596260] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1349lb # [ 0.534130] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1350lb # [ 0.536887] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1351lb # [ 0.540651] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1352worker2 # [ 0.529047] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1353worker2 # [ 0.530865] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1354client # [ 0.607675] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001355lb # [ 0.548733] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1356client # [ 0.607705] Intel/Sharp Extended Query Table at 0x00311357client # [ 0.609298] Using buffer write method1358client # [ 0.609361] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1359worker2 # [ 0.542197] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1360worker1 # [ 0.593425] loop: module loaded1361worker1 # [ 0.593620] virtio_blk virtio5: 1/0/0 default/read/poll queues1362worker1 # [ 0.594333] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1363lb # [ 0.558931] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1364client # [ 0.624220] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001365worker1 # [ 0.599256] megasas: 07.734.00.00-rc11366client # [ 0.624253] Intel/Sharp Extended Query Table at 0x00311367client # [ 0.626107] Using buffer write method1368worker1 # [ 0.599964] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1369worker2 # [ 0.552468] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1370client # [ 0.626136] Concatenating MTD devices:1371client # [ 0.626141] (0): "0.flash"1372client # [ 0.626144] (1): "0.flash"1373client # [ 0.626148] into device "0.flash"1374worker2 # [ 0.555161] msm_serial: driver initialized1375lb # [ 0.572777] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1376worker2 # [ 0.559487] SuperH (H)SCI(F) driver initialized1377worker2 # [ 0.559552] STM32 USART driver initialized1378worker1 # [ 0.611330] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001379worker1 # [ 0.611362] Intel/Sharp Extended Query Table at 0x00311380worker1 # [ 0.613030] Using buffer write method1381worker1 # [ 0.613099] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1382lb # [ 0.582483] msm_serial: driver initialized1383lb # [ 0.582688] SuperH (H)SCI(F) driver initialized1384lb # [ 0.582744] STM32 USART driver initialized1385worker1 # [ 0.627967] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001386worker1 # [ 0.628004] Intel/Sharp Extended Query Table at 0x00311387worker1 # [ 0.629622] Using buffer write method1388worker1 # [ 0.629651] Concatenating MTD devices:1389worker1 # [ 0.629655] (0): "0.flash"1390worker1 # [ 0.629659] (1): "0.flash"1391worker1 # [ 0.629662] into device "0.flash"1392worker2 # [ 0.586822] loop: module loaded1393worker2 # [ 0.586999] virtio_blk virtio5: 1/0/0 default/read/poll queues1394worker2 # [ 0.595835] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1395lb # [ 0.612086] loop: module loaded1396lb # [ 0.612271] virtio_blk virtio5: 1/0/0 default/read/poll queues1397lb # [ 0.612971] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1398worker2 # [ 0.599786] megasas: 07.734.00.00-rc11399worker2 # [ 0.600475] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1400lb # [ 0.618154] megasas: 07.734.00.00-rc11401worker2 # [ 0.604300] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402worker2 # [ 0.604353] Intel/Sharp Extended Query Table at 0x00311403lb # [ 0.618875] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1404worker2 # [ 0.605886] Using buffer write method1405lb # [ 0.621102] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406worker2 # [ 0.605945] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1407lb # [ 0.621130] Intel/Sharp Extended Query Table at 0x00311408worker2 # [ 0.615251] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409worker2 # [ 0.615276] Intel/Sharp Extended Query Table at 0x00311410worker2 # [ 0.618861] Using buffer write method1411lb # [ 0.630539] Using buffer write method1412worker2 # [ 0.618887] Concatenating MTD devices:1413worker2 # [ 0.618890] (0): "0.flash"1414worker2 # [ 0.618895] (1): "0.flash"1415lb # [ 0.630621] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1416worker2 # [ 0.618898] into device "0.flash"1417lb # [ 0.641575] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418lb # [ 0.641608] Intel/Sharp Extended Query Table at 0x00311419lb # [ 0.643265] Using buffer write method1420lb # [ 0.643298] Concatenating MTD devices:1421lb # [ 0.643302] (0): "0.flash"1422lb # [ 0.643306] (1): "0.flash"1423lb # [ 0.643310] into device "0.flash"1424client # [ 0.848027] Freeing initrd memory: 26160K1425client # [ 0.854362] tun: Universal TUN/TAP device driver, 1.61426client # [ 0.858171] thunder_xcv, ver 1.01427client # [ 0.858219] thunder_bgx, ver 1.01428client # [ 0.858242] nicpf, ver 1.01429client # [ 0.858803] e1000: Intel(R) PRO/1000 Network Driver1430client # [ 0.858810] e1000: Copyright (c) 1999-2006 Intel Corporation.1431client # [ 0.858835] e1000e: Intel(R) PRO/1000 Network Driver1432client # [ 0.858843] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1433client # [ 0.858871] igb: Intel(R) Gigabit Ethernet Network Driver1434client # [ 0.858877] igb: Copyright (c) 2007-2014 Intel Corporation.1435client # [ 0.858902] igbvf: Intel(R) Gigabit Virtual Function Network Driver1436client # [ 0.858908] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1437client # [ 0.865769] sky2: driver version 1.301438client # [ 0.867957] usbcore: registered new interface driver usb-storage1439client # [ 0.868086] usbcore: registered new interface driver usbserial_generic1440client # [ 0.868102] usbserial: USB Serial support registered for generic1441client # [ 0.868732] hv_vmbus: registering driver hyperv_keyboard1442client # [ 0.873120] ehci-pci 0000:00:0a.0: EHCI Host Controller1443client # [ 0.873155] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11444client # [ 0.873423] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001445client # [ 0.876081] rtc-pl031 9010000.pl031: registered as rtc01446client # [ 0.876114] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T09:32:56 UTC (1789896776)1447client # [ 0.876531] i2c_dev: i2c /dev entries driver1448client # [ 0.881538] sdhci: Secure Digital Host Controller Interface driver1449client # [ 0.881553] sdhci: Copyright(c) Pierre Ossman1450client # [ 0.881850] Synopsys Designware Multimedia Card Interface Driver1451client # [ 0.882244] sdhci-pltfm: SDHCI platform and OF driver helper1452client # [ 0.883046] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001453client # [ 0.883365] hub 1-0:1.0: USB hub found1454client # [ 0.883386] hub 1-0:1.0: 6 ports detected1455client # [ 0.889143] hid: raw HID events driver (C) Jiri Kosina1456worker1 # [ 0.866617] Freeing initrd memory: 26148K1457client # [ 0.889416] usbcore: registered new interface driver usbhid1458client # [ 0.889423] usbhid: USB HID core driver1459client # [ 0.892392] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1460client # [ 0.893997] drop_monitor: Initializing network drop monitor service1461client # [ 0.894134] NET: Registered PF_INET6 protocol family1462worker1 # [ 0.872610] tun: Universal TUN/TAP device driver, 1.61463client # [ 0.897216] Segment Routing with IPv61464client # [ 0.897241] In-situ OAM (IOAM) with IPv61465client # [ 0.897271] NET: Registered PF_PACKET protocol family1466client # [ 0.899003] 9pnet: Installing 9P2000 support1467client # [ 0.901830] Key type dns_resolver registered1468worker1 # [ 0.876437] thunder_xcv, ver 1.01469worker1 # [ 0.876484] thunder_bgx, ver 1.01470worker1 # [ 0.876506] nicpf, ver 1.01471worker1 # [ 0.877063] e1000: Intel(R) PRO/1000 Network Driver1472worker1 # [ 0.877070] e1000: Copyright (c) 1999-2006 Intel Corporation.1473worker1 # [ 0.877094] e1000e: Intel(R) PRO/1000 Network Driver1474worker1 # [ 0.877103] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1475worker1 # [ 0.877134] igb: Intel(R) Gigabit Ethernet Network Driver1476worker1 # [ 0.877139] igb: Copyright (c) 2007-2014 Intel Corporation.1477worker1 # [ 0.877164] igbvf: Intel(R) Gigabit Virtual Function Network Driver1478worker1 # [ 0.877169] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1479worker1 # [ 0.877305] sky2: driver version 1.301480worker1 # [ 0.879248] ehci-pci 0000:00:0a.0: EHCI Host Controller1481client # [ 0.908338] registered taskstats version 11482client # [ 0.908496] Loading compiled-in X.509 certificates1483worker1 # [ 0.879280] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11484worker1 # [ 0.879479] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001485worker1 # [ 0.889314] usbcore: registered new interface driver usb-storage1486worker1 # [ 0.889413] usbcore: registered new interface driver usbserial_generic1487worker1 # [ 0.889427] usbserial: USB Serial support registered for generic1488worker1 # [ 0.890031] hv_vmbus: registering driver hyperv_keyboard1489client # [ 0.917545] Demotion targets for Node 0: null1490client # [ 0.917709] Key type .fscrypt registered1491worker1 # [ 0.890683] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001492client # [ 0.917715] Key type fscrypt-provisioning registered1493worker1 # [ 0.891028] hub 1-0:1.0: USB hub found1494worker1 # [ 0.891049] hub 1-0:1.0: 6 ports detected1495client # [ 0.917822] ima: No TPM chip found, activating TPM-bypass!1496client # [ 0.917843] ima: Allocated hash algorithm: sha11497client # [ 0.917871] ima: No architecture policies found1498worker1 # [ 0.896282] rtc-pl031 9010000.pl031: registered as rtc01499client # [ 0.922175] input: gpio-keys as /devices/platform/gpio-keys/input/input01500worker1 # [ 0.896311] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T09:32:56 UTC (1789896776)1501worker1 # [ 0.896640] i2c_dev: i2c /dev entries driver1502worker1 # [ 0.901666] sdhci: Secure Digital Host Controller Interface driver1503lb # [ 0.863084] Freeing initrd memory: 26140K1504worker1 # [ 0.901678] sdhci: Copyright(c) Pierre Ossman1505worker1 # [ 0.901948] Synopsys Designware Multimedia Card Interface Driver1506worker1 # [ 0.902327] sdhci-pltfm: SDHCI platform and OF driver helper1507lb # [ 0.869372] tun: Universal TUN/TAP device driver, 1.61508worker1 # [ 0.906770] hid: raw HID events driver (C) Jiri Kosina1509worker1 # [ 0.907032] usbcore: registered new interface driver usbhid1510worker1 # [ 0.907038] usbhid: USB HID core driver1511worker1 # [ 0.909980] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1512lb # [ 0.873294] thunder_xcv, ver 1.01513lb # [ 0.873347] thunder_bgx, ver 1.01514lb # [ 0.873369] nicpf, ver 1.01515worker1 # [ 0.912626] drop_monitor: Initializing network drop monitor service1516worker2 # [ 0.860027] Freeing initrd memory: 26148K1517worker1 # [ 0.912775] NET: Registered PF_INET6 protocol family1518worker1 # [ 0.914903] Segment Routing with IPv61519worker1 # [ 0.914923] In-situ OAM (IOAM) with IPv61520worker1 # [ 0.914952] NET: Registered PF_PACKET protocol family1521lb # [ 0.875141] e1000: Intel(R) PRO/1000 Network Driver1522worker2 # [ 0.866047] tun: Universal TUN/TAP device driver, 1.61523lb # [ 0.875153] e1000: Copyright (c) 1999-2006 Intel Corporation.1524lb # [ 0.875179] e1000e: Intel(R) PRO/1000 Network Driver1525client # [ 0.941072] clk: Disabling unused clocks1526worker1 # [ 0.916592] 9pnet: Installing 9P2000 support1527lb # [ 0.875189] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1528client # [ 0.941111] PM: genpd: Disabling unused power domains1529lb # [ 0.875218] igb: Intel(R) Gigabit Ethernet Network Driver1530lb # [ 0.875223] igb: Copyright (c) 2007-2014 Intel Corporation.1531lb # [ 0.875247] igbvf: Intel(R) Gigabit Virtual Function Network Driver1532lb # [ 0.875252] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1533client # [ 0.945412] Freeing unused kernel memory: 4736K1534worker2 # [ 0.869763] thunder_xcv, ver 1.01535lb # [ 0.875386] sky2: driver version 1.301536client # [ 0.945623] Run /init as init process1537worker2 # [ 0.869807] thunder_bgx, ver 1.01538worker2 # [ 0.869831] nicpf, ver 1.01539worker1 # [ 0.919276] Key type dns_resolver registered1540lb # [ 0.877033] usbcore: registered new interface driver usb-storage1541worker2 # [ 0.870384] e1000: Intel(R) PRO/1000 Network Driver1542lb # [ 0.877116] usbcore: registered new interface driver usbserial_generic1543worker2 # [ 0.870392] e1000: Copyright (c) 1999-2006 Intel Corporation.1544lb # [ 0.877130] usbserial: USB Serial support registered for generic1545worker2 # [ 0.870420] e1000e: Intel(R) PRO/1000 Network Driver1546lb # [ 0.878173] ehci-pci 0000:00:0a.0: EHCI Host Controller1547worker2 # [ 0.870428] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1548lb # [ 0.878198] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11549worker2 # [ 0.870455] igb: Intel(R) Gigabit Ethernet Network Driver1550lb # [ 0.878392] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001551worker2 # [ 0.870461] igb: Copyright (c) 2007-2014 Intel Corporation.1552worker1 # [ 0.925768] registered taskstats version 11553worker1 # [ 0.925915] Loading compiled-in X.509 certificates1554worker2 # [ 0.870484] igbvf: Intel(R) Gigabit Virtual Function Network Driver1555lb # [ 0.889031] hv_vmbus: registering driver hyperv_keyboard1556worker2 # [ 0.870490] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1557worker2 # [ 0.870624] sky2: driver version 1.301558lb # [ 0.890213] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001559lb # [ 0.891274] hub 1-0:1.0: USB hub found1560lb # [ 0.891751] hub 1-0:1.0: 6 ports detected1561worker2 # [ 0.879820] ehci-pci 0000:00:0a.0: EHCI Host Controller1562worker2 # [ 0.879851] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11563lb # [ 0.892989] rtc-pl031 9010000.pl031: registered as rtc01564worker2 # [ 0.880006] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001565lb # [ 0.893013] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T09:32:57 UTC (1789896777)1566lb # [ 0.893333] i2c_dev: i2c /dev entries driver1567worker2 # [ 0.882488] usbcore: registered new interface driver usb-storage1568worker1 # [ 0.934386] Demotion targets for Node 0: null1569worker2 # [ 0.882542] usbcore: registered new interface driver usbserial_generic1570worker1 # [ 0.934494] Key type .fscrypt registered1571worker2 # [ 0.882557] usbserial: USB Serial support registered for generic1572worker1 # [ 0.934500] Key type fscrypt-provisioning registered1573worker2 # [ 0.883214] hv_vmbus: registering driver hyperv_keyboard1574worker1 # [ 0.934592] ima: No TPM chip found, activating TPM-bypass!1575worker1 # [ 0.934611] ima: Allocated hash algorithm: sha11576client # [ 0.961468] systemd[1]: Successfully made /usr/ read-only.1577worker1 # [ 0.934633] ima: No architecture policies found1578worker2 # [ 0.887005] rtc-pl031 9010000.pl031: registered as rtc01579worker2 # [ 0.887036] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T09:32:57 UTC (1789896777)1580worker1 # [ 0.938910] input: gpio-keys as /devices/platform/gpio-keys/input/input01581lb # [ 0.898245] sdhci: Secure Digital Host Controller Interface driver1582lb # [ 0.898255] sdhci: Copyright(c) Pierre Ossman1583worker2 # [ 0.889133] i2c_dev: i2c /dev entries driver1584lb # [ 0.898526] Synopsys Designware Multimedia Card Interface Driver1585lb # [ 0.898896] sdhci-pltfm: SDHCI platform and OF driver helper1586lb # [ 0.900631] hid: raw HID events driver (C) Jiri Kosina1587lb # [ 0.900872] usbcore: registered new interface driver usbhid1588lb # [ 0.900878] usbhid: USB HID core driver1589worker2 # [ 0.892536] sdhci: Secure Digital Host Controller Interface driver1590worker2 # [ 0.892547] sdhci: Copyright(c) Pierre Ossman1591lb # [ 0.906492] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1592worker2 # [ 0.892815] Synopsys Designware Multimedia Card Interface Driver1593lb # [ 0.908061] drop_monitor: Initializing network drop monitor service1594worker2 # [ 0.893194] sdhci-pltfm: SDHCI platform and OF driver helper1595lb # [ 0.908240] NET: Registered PF_INET6 protocol family1596worker2 # [ 0.894905] hid: raw HID events driver (C) Jiri Kosina1597worker2 # [ 0.895154] usbcore: registered new interface driver usbhid1598lb # [ 0.911311] Segment Routing with IPv61599worker2 # [ 0.895161] usbhid: USB HID core driver1600lb # [ 0.911331] In-situ OAM (IOAM) with IPv61601lb # [ 0.911362] NET: Registered PF_PACKET protocol family1602worker2 # [ 0.899739] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001603worker2 # [ 0.900051] hub 1-0:1.0: USB hub found1604lb # [ 0.912952] 9pnet: Installing 9P2000 support1605worker2 # [ 0.900072] hub 1-0:1.0: 6 ports detected1606worker2 # [ 0.903124] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1607lb # [ 0.915756] Key type dns_resolver registered1608worker2 # [ 0.905742] drop_monitor: Initializing network drop monitor service1609worker2 # [ 0.905886] NET: Registered PF_INET6 protocol family1610worker2 # [ 0.907798] Segment Routing with IPv61611worker1 # [ 0.957387] clk: Disabling unused clocks1612worker2 # [ 0.907816] In-situ OAM (IOAM) with IPv61613worker1 # [ 0.957411] PM: genpd: Disabling unused power domains1614worker2 # [ 0.907847] NET: Registered PF_PACKET protocol family1615worker2 # [ 0.909485] 9pnet: Installing 9P2000 support1616worker1 # [ 0.961657] Freeing unused kernel memory: 4736K1617worker1 # [ 0.961880] Run /init as init process1618lb # [ 0.922181] registered taskstats version 11619lb # [ 0.922333] Loading compiled-in X.509 certificates1620worker2 # [ 0.912249] Key type dns_resolver registered1621lb # [ 0.930698] Demotion targets for Node 0: null1622lb # [ 0.930803] Key type .fscrypt registered1623worker2 # [ 0.918450] registered taskstats version 11624lb # [ 0.930809] Key type fscrypt-provisioning registered1625worker2 # [ 0.918608] Loading compiled-in X.509 certificates1626lb # [ 0.930906] ima: No TPM chip found, activating TPM-bypass!1627lb # [ 0.930926] ima: Allocated hash algorithm: sha11628lb # [ 0.930946] ima: No architecture policies found1629lb # [ 0.935000] input: gpio-keys as /devices/platform/gpio-keys/input/input01630worker1 # [ 0.976879] systemd[1]: Successfully made /usr/ read-only.1631worker2 # [ 0.927142] Demotion targets for Node 0: null1632worker2 # [ 0.927967] Key type .fscrypt registered1633worker2 # [ 0.927977] Key type fscrypt-provisioning registered1634worker2 # [ 0.928071] ima: No TPM chip found, activating TPM-bypass!1635worker2 # [ 0.928091] ima: Allocated hash algorithm: sha11636worker2 # [ 0.928114] ima: No architecture policies found1637worker2 # [ 0.931538] input: gpio-keys as /devices/platform/gpio-keys/input/input01638lb # [ 0.952951] clk: Disabling unused clocks1639lb # [ 0.952980] PM: genpd: Disabling unused power domains1640lb # [ 0.957161] Freeing unused kernel memory: 4736K1641lb # [ 0.957369] Run /init as init process1642worker2 # [ 0.949447] clk: Disabling unused clocks1643worker2 # [ 0.949474] PM: genpd: Disabling unused power domains1644worker2 # [ 0.953648] Freeing unused kernel memory: 4736K1645worker2 # [ 0.953850] Run /init as init process1646lb # [ 0.971799] systemd[1]: Successfully made /usr/ read-only.1647worker2 # [ 0.969483] systemd[1]: Successfully made /usr/ read-only.1648client # [ 1.135088] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker1 # [ 1.138723] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.137594] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker2 # [ 1.147299] usb 1-1: new high-speed USB device number 2 using ehci-pci1652client # [ 1.287296] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653client # [ 1.296374] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654client # [ 1.308678] systemd[1]: Detected virtualization qemu.1655client # [ 1.310736] systemd[1]: Detected architecture arm64.1656client # [ 1.312696] systemd[1]: Running in initrd.1657client # [ 1.315262] systemd[1]: Initializing machine ID from random generator.1658client # [ 1.318061] systemd[1]: Hostname set to <client>.1659worker1 # [ 1.290979] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660worker1 # [ 1.311729] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1661worker1 # [ 1.323573] systemd[1]: Detected virtualization qemu.1662worker1 # [ 1.325526] systemd[1]: Detected architecture arm64.1663worker1 # [ 1.327456] systemd[1]: Running in initrd.1664worker1 # [ 1.330010] systemd[1]: Initializing machine ID from random generator.1665lb # [ 1.289907] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11666worker1 # [ 1.332998] systemd[1]: Hostname set to <worker1>.1667client # [ 1.375283] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01668worker2 # [ 1.299518] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11669lb # [ 1.306567] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1670lb # [ 1.318517] systemd[1]: Detected virtualization qemu.1671lb # [ 1.320489] systemd[1]: Detected architecture arm64.1672lb # [ 1.322431] systemd[1]: Running in initrd.1673lb # [ 1.324978] systemd[1]: Initializing machine ID from random generator.1674lb # [ 1.327852] systemd[1]: Hostname set to <lb>.1675worker2 # [ 1.305531] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1676worker2 # [ 1.305594] systemd[1]: Detected virtualization qemu.1677worker2 # [ 1.305665] systemd[1]: Detected architecture arm64.1678worker2 # [ 1.305690] systemd[1]: Running in initrd.1679worker2 # [ 1.306597] systemd[1]: Initializing machine ID from random generator.1680worker2 # [ 1.306897] systemd[1]: Hostname set to <worker2>.1681worker1 # [ 1.378925] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.381810] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker2 # [ 1.387475] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01684client # [ 1.495067] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker1 # [ 1.498734] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.501579] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker2 # [ 1.507274] usb 1-2: new high-speed USB device number 3 using ehci-pci1688client # [ 1.622591] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689client # [ 1.651508] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21690client # [ 1.656541] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01691worker1 # [ 1.635068] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692worker1 # [ 1.661082] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693worker1 # [ 1.666745] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694worker2 # [ 1.630381] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695lb # [ 1.651654] systemd[1]: bpf-restrict-fs: LSM BPF program attached1696lb # [ 1.661814] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21697lb # [ 1.667389] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01698client # [ 1.737134] systemd[1]: Queued start job for default target Initrd Default Target.1699worker2 # [ 1.661637] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21700worker2 # [ 1.666295] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01701client # [ 1.745348] systemd[1]: Created slice Slice /system/modprobe.1702client # [ 1.746553] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1703client # [ 1.747955] systemd[1]: Expecting device /dev/disk/by-label/nixos...1704client # [ 1.749005] systemd[1]: Reached target Path Units.1705client # [ 1.749840] systemd[1]: Reached target Slice Units.1706client # [ 1.750675] systemd[1]: Reached target Swaps.1707client # [ 1.751496] systemd[1]: Reached target Timer Units.1708client # [ 1.752527] systemd[1]: Listening on D-Bus System Message Bus Socket.1709client # [ 1.753756] systemd[1]: Listening on Journal Socket (/dev/log).1710client # [ 1.754858] systemd[1]: Listening on Journal Sockets.1711client # [ 1.755891] systemd[1]: Listening on udev Control Socket.1712client # [ 1.756013] systemd[1]: Listening on udev Kernel Socket.1713client # [ 1.756036] systemd[1]: Reached target Socket Units.1714client # [ 1.760169] systemd[1]: Starting Create List of Static Device Nodes...1715client # [ 1.770463] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1716client # [ 1.771769] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1717worker1 # [ 1.752050] systemd[1]: Queued start job for default target Initrd Default Target.1718worker1 # [ 1.761061] systemd[1]: Created slice Slice /system/modprobe.1719client # [ 1.784500] systemd[1]: Mounting Kernel Configuration File System...1720worker1 # [ 1.762272] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1721worker1 # [ 1.763705] systemd[1]: Expecting device /dev/disk/by-label/nixos...1722worker1 # [ 1.764757] systemd[1]: Reached target Path Units.1723worker1 # [ 1.765579] systemd[1]: Reached target Slice Units.1724worker1 # [ 1.766404] systemd[1]: Reached target Swaps.1725worker1 # [ 1.767212] systemd[1]: Reached target Timer Units.1726worker1 # [ 1.768291] systemd[1]: Listening on D-Bus System Message Bus Socket.1727worker1 # [ 1.769524] systemd[1]: Listening on Journal Socket (/dev/log).1728worker1 # [ 1.770667] systemd[1]: Listening on Journal Sockets.1729worker1 # [ 1.771665] systemd[1]: Listening on udev Control Socket.1730worker1 # [ 1.772680] systemd[1]: Listening on udev Kernel Socket.1731worker1 # [ 1.773582] systemd[1]: Reached target Socket Units.1732worker1 # [ 1.776093] systemd[1]: Starting Create List of Static Device Nodes...1733client # [ 1.807219] systemd[1]: Starting Journal Service...1734worker1 # [ 1.786440] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1735client # [ 1.811223] systemd[1]: Starting Load Kernel Modules...1736client # [ 1.812145] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1737worker1 # [ 1.787806] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1738client # [ 1.816202] systemd[1]: Starting Coldplug All udev Devices...1739worker2 # [ 1.745659] systemd[1]: Queued start job for default target Initrd Default Target.1740lb # [ 1.759124] systemd[1]: Queued start job for default target Initrd Default Target.1741client # [ 1.825835] systemd[1]: Finished Create List of Static Device Nodes.1742worker1 # [ 1.800140] systemd[1]: Mounting Kernel Configuration File System...1743client # [ 1.827652] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1744lb # [ 1.767782] systemd[1]: Created slice Slice /system/modprobe.1745lb # [ 1.768912] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1746worker2 # [ 1.757678] systemd[1]: Created slice Slice /system/modprobe.1747lb # [ 1.770238] systemd[1]: Expecting device /dev/disk/by-label/nixos...1748lb # [ 1.771269] systemd[1]: Reached target Path Units.1749lb # [ 1.772054] systemd[1]: Reached target Slice Units.1750worker2 # [ 1.758936] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1751lb # [ 1.772857] systemd[1]: Reached target Swaps.1752worker2 # [ 1.760470] systemd[1]: Expecting device /dev/disk/by-label/nixos...1753lb # [ 1.773615] systemd[1]: Reached target Timer Units.1754worker2 # [ 1.761583] systemd[1]: Reached target Path Units.1755lb # [ 1.774566] systemd[1]: Listening on D-Bus System Message Bus Socket.1756worker2 # [ 1.762430] systemd[1]: Reached target Slice Units.1757worker2 # [ 1.763338] systemd[1]: Reached target Swaps.1758lb # [ 1.775765] systemd[1]: Listening on Journal Socket (/dev/log).1759worker2 # [ 1.764122] systemd[1]: Reached target Timer Units.1760lb # [ 1.776854] systemd[1]: Listening on Journal Sockets.1761worker2 # [ 1.765175] systemd[1]: Listening on D-Bus System Message Bus Socket.1762worker2 # [ 1.766453] systemd[1]: Listening on Journal Socket (/dev/log).1763lb # [ 1.777825] systemd[1]: Listening on udev Control Socket.1764lb # [ 1.777940] systemd[1]: Listening on udev Kernel Socket.1765lb # [ 1.777963] systemd[1]: Reached target Socket Units.1766worker2 # [ 1.767720] systemd[1]: Listening on Journal Sockets.1767client # [ 1.843467] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1768lb # [ 1.782067] systemd[1]: Starting Create List of Static Device Nodes...1769worker2 # [ 1.767871] systemd[1]: Listening on udev Control Socket.1770client # [ 1.845821] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1771worker2 # [ 1.767997] systemd[1]: Listening on udev Kernel Socket.1772worker2 # [ 1.768020] systemd[1]: Reached target Socket Units.1773worker1 # [ 1.822788] systemd[1]: Starting Journal Service...1774worker2 # [ 1.773238] systemd[1]: Starting Create List of Static Device Nodes...1775worker1 # [ 1.826856] systemd[1]: Starting Load Kernel Modules...1776lb # [ 1.789672] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1777worker1 # [ 1.827776] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1778lb # [ 1.790788] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1779worker2 # [ 1.780181] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1780worker1 # [ 1.831932] systemd[1]: Starting Coldplug All udev Devices...1781worker2 # [ 1.781396] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1782lb # [ 1.796262] systemd[1]: Mounting Kernel Configuration File System...1783worker2 # [ 1.791419] systemd[1]: Mounting Kernel Configuration File System...1784worker1 # [ 1.842954] systemd[1]: Finished Create List of Static Device Nodes.1785client # [ 1.867315] systemd[1]: Mounted Kernel Configuration File System.1786lb # [ 1.806107] systemd[1]: Starting Journal Service...1787worker1 # [ 1.844756] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1788worker2 # [ 1.800869] systemd[1]: Starting Journal Service...1789client # [ 1.878505] systemd-journald[73]: Collecting audit messages is disabled.1790client # [ 1.883456] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1791worker1 # [ 1.858895] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1792client # [ 1.899650] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1793lb # [ 1.837794] systemd[1]: Starting Load Kernel Modules...1794client # [ 1.901615] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1795lb # [ 1.837923] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1796worker2 # [ 1.827451] systemd[1]: Starting Load Kernel Modules...1797worker2 # [ 1.827562] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1798worker1 # [ 1.880422] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1799client # [ 1.903432] systemd[1]: Starting Create Static Device Nodes in /dev...1800lb # [ 1.848860] systemd[1]: Starting Coldplug All udev Devices...1801client # [ 1.912196] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01802client # [ 1.912424] [drm] features: -virgl +edid -resource_blob -host_visible1803client # [ 1.912434] [drm] features: -context_init1804client # [ 1.913150] [drm] number of scanouts: 11805client # [ 1.913167] [drm] number of cap sets: 01806worker1 # [ 1.893647] systemd-journald[73]: Collecting audit messages is disabled.1807lb # [ 1.856832] systemd-journald[73]: Collecting audit messages is disabled.1808worker1 # [ 1.895170] systemd[1]: Mounted Kernel Configuration File System.1809worker1 # [ 1.898433] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1810worker2 # [ 1.855480] systemd[1]: Starting Coldplug All udev Devices...1811lb # [ 1.867134] systemd[1]: Finished Create List of Static Device Nodes.1812lb # [ 1.868052] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1813lb # [ 1.868342] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1814lb # [ 1.868570] systemd[1]: Mounted Kernel Configuration File System.1815client # [ 1.931577] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1816client # [ 1.931601] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01817worker1 # [ 1.911658] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1818worker2 # [ 1.863402] systemd[1]: Finished Create List of Static Device Nodes.1819worker1 # [ 1.916523] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1820worker2 # [ 1.864275] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1821worker2 # [ 1.864559] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1822worker2 # [ 1.864789] systemd[1]: Mounted Kernel Configuration File System.1823worker2 # [ 1.872546] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1824worker1 # [ 1.924363] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01825worker1 # [ 1.924600] [drm] features: -virgl +edid -resource_blob -host_visible1826worker2 # [ 1.875714] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1827client # [ 1.951215] systemd[1]: Finished Create Static Device Nodes in /dev.1828worker1 # [ 1.924610] [drm] features: -context_init1829worker1 # [ 1.925331] [drm] number of scanouts: 11830lb # [ 1.885862] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1831client # [ 1.952209] systemd[1]: Reached target Preparation for Local File Systems.1832worker1 # [ 1.925364] [drm] number of cap sets: 01833client # [ 1.953075] systemd[1]: Reached target Local File Systems.1834worker1 # [ 1.929099] systemd[1]: Starting Create Static Device Nodes in /dev...1835worker2 # [ 1.883313] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1836worker1 # [ 1.938049] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1837worker1 # [ 1.938073] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01838worker2 # [ 1.891163] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01839client # [ 1.963400] systemd[1]: Starting Rule-based Manager for Device Events and Files...1840lb # [ 1.908320] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1841worker2 # [ 1.897913] systemd-journald[73]: Collecting audit messages is disabled.1842client # [ 1.968260] Console: switching to colour frame buffer device 160x501843client # [ 1.974911] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1844worker2 # [ 1.907460] [drm] features: -virgl +edid -resource_blob -host_visible1845worker2 # [ 1.907478] [drm] features: -context_init1846worker2 # [ 1.908222] [drm] number of scanouts: 11847worker2 # [ 1.908240] [drm] number of cap sets: 01848lb # [ 1.922115] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1849lb # [ 1.925890] systemd[1]: Starting Create Static Device Nodes in /dev...1850worker1 # [ 1.967551] systemd[1]: Finished Create Static Device Nodes in /dev.1851worker1 # [ 1.967732] systemd[1]: Reached target Preparation for Local File Systems.1852worker1 # [ 1.967758] systemd[1]: Reached target Local File Systems.1853lb # [ 1.933696] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1854worker2 # [ 1.916383] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1855worker2 # [ 1.916407] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01856worker1 # [ 1.971700] systemd[1]: Starting Rule-based Manager for Device Events and Files...1857client # [ 2.003586] systemd[1]: Finished Load Kernel Modules.1858lb # [ 1.941957] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01859lb # [ 1.942185] [drm] features: -virgl +edid -resource_blob -host_visible1860lb # [ 1.942195] [drm] features: -context_init1861lb # [ 1.943005] [drm] number of scanouts: 11862worker1 # [ 1.976109] Console: switching to colour frame buffer device 160x501863lb # [ 1.943023] [drm] number of cap sets: 01864worker1 # [ 1.982543] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1865client # [ 2.012255] systemd[1]: Starting Apply Kernel Variables...1866worker2 # [ 1.939928] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1867lb # [ 1.957843] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1868lb # [ 1.957864] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01869worker2 # [ 1.943539] Console: switching to colour frame buffer device 160x501870worker2 # [ 1.951625] systemd[1]: Starting Create Static Device Nodes in /dev...1871worker1 # [ 2.015285] systemd[1]: Finished Load Kernel Modules.1872worker1 # [ 2.018437] systemd[1]: Starting Apply Kernel Variables...1873worker2 # [ 1.967853] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874lb # [ 1.989625] systemd[1]: Finished Create Static Device Nodes in /dev.1875lb # [ 1.989975] systemd[1]: Reached target Preparation for Local File Systems.1876lb # [ 1.990002] systemd[1]: Reached target Local File Systems.1877worker2 # [ 1.984512] systemd[1]: Finished Load Kernel Modules.1878lb # [ 1.993663] Console: switching to colour frame buffer device 160x501879worker2 # [ 1.987570] systemd[1]: Starting Apply Kernel Variables...1880lb # [ 2.001483] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881client # [ 2.067651] systemd[1]: Finished Apply Kernel Variables.1882client # [ 2.061030] systemd-modules-load[74]: Inserted module 'dm_mod'1883client # [ 2.077100] systemd[1]: Started Journal Service.1884client # [ 2.065517] systemd-modules-load[74]: Module 'virtio_balloon' is built in1885lb # [ 2.018231] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1886worker2 # [ 2.005552] systemd[1]: Finished Create Static Device Nodes in /dev.1887worker2 # [ 2.007913] systemd[1]: Reached target Preparation for Local File Systems.1888worker2 # [ 2.009973] systemd[1]: Reached target Local File Systems.1889client # [ 2.072249] systemd-modules-load[74]: Module 'virtio_console' is built in1890client # [ 2.076324] systemd-modules-load[74]: Inserted module 'virtio_gpu'1891client # [ 2.077342] systemd-modules-load[74]: Module 'virtio_rng' is built in1892lb # [ 2.030771] systemd[1]: Finished Load Kernel Modules.1893client # [ 2.080531] systemd-udevd[80]: Using default interface naming scheme 'v261'.1894worker1 # [ 2.071327] systemd[1]: Finished Apply Kernel Variables.1895worker2 # [ 2.024693] systemd[1]: Starting Rule-based Manager for Device Events and Files...1896client # [ 2.088417] systemd[1]: Starting Create System Files and Directories...1897lb # [ 2.040540] systemd[1]: Starting Apply Kernel Variables...1898worker1 # [ 2.088364] systemd[1]: Started Journal Service.1899lb # [ 2.036349] systemd-modules-load[74]: Inserted module 'dm_mod'1900lb # [ 2.038376] systemd-modules-load[74]: Module 'virtio_balloon' is built in1901worker1 # [ 2.080325] systemd-modules-load[74]: Inserted module 'dm_mod'1902lb # [ 2.039499] systemd-modules-load[74]: Module 'virtio_console' is built in1903worker1 # [ 2.081587] systemd-modules-load[74]: Module 'virtio_balloon' is built in1904worker1 # [ 2.082800] systemd-modules-load[74]: Module 'virtio_console' is built in1905worker1 # [ 2.083839] systemd-modules-load[74]: Inserted module 'virtio_gpu'1906client # [ 2.107791] systemd[1]: Finished Create System Files and Directories.1907lb # [ 2.065745] systemd[1]: Started Journal Service.1908worker1 # [ 2.093083] systemd-modules-load[74]: Module 'virtio_rng' is built in1909lb # [ 2.052178] systemd-modules-load[74]: Inserted module 'virtio_gpu'1910worker1 # [ 2.094427] systemd-udevd[80]: Using default interface naming scheme 'v261'.1911worker1 # [ 2.095549] systemd[1]: Starting Create System Files and Directories...1912worker2 # [ 2.059533] systemd[1]: Finished Apply Kernel Variables.1913lb # [ 2.060411] systemd-modules-load[74]: Module 'virtio_rng' is built in1914client # [ 2.127293] systemd[1]: Started Rule-based Manager for Device Events and Files.1915worker2 # [ 2.070998] systemd[1]: Started Journal Service.1916lb # [ 2.068079] systemd[1]: Starting Create System Files and Directories...1917lb # [ 2.077081] systemd[1]: Finished Apply Kernel Variables.1918worker2 # [ 2.068266] systemd-modules-load[74]: Inserted module 'dm_mod'1919worker2 # [ 2.072251] systemd-modules-load[74]: Module 'virtio_balloon' is built in1920worker2 # [ 2.073371] systemd-modules-load[74]: Module 'virtio_console' is built in1921lb # [ 2.084345] systemd-udevd[80]: Using default interface naming scheme 'v261'.1922worker2 # [ 2.074421] systemd-modules-load[74]: Inserted module 'virtio_gpu'1923worker1 # [ 2.125609] systemd[1]: Finished Create System Files and Directories.1924worker2 # [ 2.075355] systemd-modules-load[74]: Module 'virtio_rng' is built in1925worker2 # [ 2.085217] systemd[1]: Starting Create System Files and Directories...1926worker1 # [ 2.136911] systemd[1]: Started Rule-based Manager for Device Events and Files.1927lb # [ 2.107534] systemd[1]: Finished Create System Files and Directories.1928client # [ 2.179800] systemd[1]: Starting Virtual Console Setup...1929worker2 # [ 2.112500] systemd-udevd[81]: Using default interface naming scheme 'v261'.1930worker2 # [ 2.116220] systemd[1]: Finished Create System Files and Directories.1931lb # [ 2.128873] systemd[1]: Started Rule-based Manager for Device Events and Files.1932worker1 # [ 2.192605] systemd[1]: Starting Virtual Console Setup...1933worker2 # [ 2.145523] systemd[1]: Started Rule-based Manager for Device Events and Files.1934client # [ 2.224435] systemd-vconsole-setup[104]: Configuration of first virtual console was skipped, ignoring remaining ones.1935client # [ 2.227838] systemd[1]: Finished Virtual Console Setup.1936lb # [ 2.177969] systemd[1]: Starting Virtual Console Setup...1937worker1 # [ 2.248471] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938worker1 # [ 2.251883] systemd[1]: Finished Virtual Console Setup.1939worker2 # [ 2.203905] systemd[1]: Starting Virtual Console Setup...1940lb # [ 2.228440] systemd-vconsole-setup[104]: Configuration of first virtual console was skipped, ignoring remaining ones.1941lb # [ 2.231679] systemd[1]: Finished Virtual Console Setup.1942worker2 # [ 2.252460] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1943worker2 # [ 2.255719] systemd[1]: Finished Virtual Console Setup.1944client # [ 2.823591] systemd[1]: Finished Coldplug All udev Devices.1945client # [ 2.824614] systemd[1]: Reached target System Initialization.1946client # [ 2.825421] systemd[1]: Reached target Basic System.1947lb # [ 2.827225] systemd[1]: Finished Coldplug All udev Devices.1948worker1 # [ 2.869188] systemd[1]: Finished Coldplug All udev Devices.1949lb # [ 2.828199] systemd[1]: Reached target System Initialization.1950lb # [ 2.829002] systemd[1]: Reached target Basic System.1951worker1 # [ 2.870166] systemd[1]: Reached target System Initialization.1952worker1 # [ 2.871165] systemd[1]: Reached target Basic System.1953worker2 # [ 2.859317] systemd[1]: Finished Coldplug All udev Devices.1954worker2 # [ 2.860496] systemd[1]: Reached target System Initialization.1955worker2 # [ 2.864097] systemd[1]: Reached target Basic System.1956client # [ 2.968758] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957client # [ 2.995937] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1958client # [ 3.003646] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1959lb # [ 2.983719] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960lb # [ 2.988260] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1961worker1 # [ 3.037365] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1962worker1 # [ 3.040938] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1963client # [ 3.069257] systemd[1]: Found device /dev/disk/by-label/nixos.1964lb # [ 3.004755] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1965worker1 # [ 3.049448] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1966client # [ 3.076122] systemd[1]: Reached target Initrd Root Device.1967client # [ 3.081321] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1968worker2 # [ 3.031186] (udev-worker)[107]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969client # [ 3.132003] systemd-fsck[113]: nixos: clean, 12/65536 files, 13019/262144 blocks1970worker2 # [ 3.061537] (udev-worker)[107]: Network interface NamePolicy= disabled on kernel command line.1971lb # [ 3.072397] systemd[1]: Found device /dev/disk/by-label/nixos.1972client # [ 3.138305] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1973worker2 # [ 3.065114] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 3.143062] systemd[1]: Mounting /sysroot...1975lb # [ 3.078614] systemd[1]: Reached target Initrd Root Device.1976lb # [ 3.082224] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1977worker1 # [ 3.124491] systemd[1]: Found device /dev/disk/by-label/nixos.1978worker1 # [ 3.125430] systemd[1]: Reached target Initrd Root Device.1979worker1 # [ 3.129636] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980client # [ 3.189649] EXT4-fs (vda): mounted filesystem 245aedea-c598-4d60-a5fa-ba226cc559af r/w with ordered data mode. Quota mode: none.1981client # [ 3.181392] systemd[1]: Mounted /sysroot.1982client # [ 3.185251] systemd[1]: Reached target Initrd Root File System.1983client # [ 3.186689] systemd[1]: Starting Mountpoints Configured in the Real Root...1984lb # [ 3.125151] systemd-fsck[113]: nixos: clean, 12/65536 files, 13019/262144 blocks1985worker1 # [ 3.173065] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1986lb # [ 3.132077] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1987lb # [ 3.134057] systemd[1]: Mounting /sysroot...1988worker2 # [ 3.127205] systemd[1]: Found device /dev/disk/by-label/nixos.1989worker1 # [ 3.180164] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990worker2 # [ 3.130536] systemd[1]: Reached target Initrd Root Device.1991worker1 # [ 3.182587] systemd[1]: Mounting /sysroot...1992worker2 # [ 3.136148] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1993client # [ 3.211524] systemd-sysroot-fstab-check[121]: /sysroot should be mounted in the initrd, will request daemon-reload.1994client # [ 3.216274] systemd[1]: Reload requested from client PID 121 ('systemd-sysroot') (unit initrd-parse-etc.service)...1995client # [ 3.220993] systemd[1]: Reloading...1996lb # [ 3.179504] EXT4-fs (vda): mounted filesystem f5afaaae-8b76-4c33-87b6-c37da8610163 r/w with ordered data mode. Quota mode: none.1997lb # [ 3.166991] systemd[1]: Mounted /sysroot.1998lb # [ 3.169203] systemd[1]: Reached target Initrd Root File System.1999lb # [ 3.176113] systemd[1]: Starting Mountpoints Configured in the Real Root...2000worker1 # [ 3.232320] EXT4-fs (vda): mounted filesystem d68d2f67-b566-43cd-a5a9-f5e3e0f2d05d r/w with ordered data mode. Quota mode: none.
loading 1,412 hidden lines…
3413lb # [ 17.127118] postgres[801]: [801] LOG: listening on IPv4 address "127.0.0.1", port 54323414lb # [ 17.128981] postgres[801]: [801] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3415lb # [ 17.141585] postgres[817]: [817] LOG: database system was shut down at 2026-09-20 09:33:13 GMT3416lb # [ 17.148321] postgres[801]: [801] LOG: database system is ready to accept connections3417lb # [ 17.152818] systemd[1]: Started PostgreSQL Server.3418lb # [ 17.160115] systemd[1]: Starting PostgreSQL Setup Scripts...3419lb # [ 17.382411] postgresql-setup-start[828]: CREATE DATABASE3420lb # [ 17.432935] postgresql-setup-start[833]: CREATE ROLE3421lb # [ 17.457668] postgresql-setup-start[835]: ALTER DATABASE3422lb # [ 17.464568] systemd[1]: Finished PostgreSQL Setup Scripts.3423lb # [ 17.466481] systemd[1]: Reached target PostgreSQL.3424worker1 # [ 18.175295] dhcpcd[642]: eth0: leased 10.0.2.15 for 86400 seconds3425worker1 # [ 18.179379] dhcpcd[642]: eth0: adding route to 10.0.2.0/243426worker1 # [ 18.182879] dhcpcd[642]: eth0: adding default route via 10.0.2.23427lb # [ 18.170844] dhcpcd[665]: eth0: leased 10.0.2.15 for 86400 seconds3428lb # [ 18.171040] dhcpcd[665]: eth0: adding route to 10.0.2.0/243429lb # [ 18.171118] dhcpcd[665]: eth0: adding default route via 10.0.2.23430worker1 # [ 18.354714] systemd[1]: Started DHCP Client.3431worker1 # [ 18.356895] systemd[1]: Reached target Multi-User System.3432worker1 # [ 18.357755] systemd[1]: Startup finished in 951ms (kernel) + 5.456s (initrd) + 11.948s (userspace) = 18.356s.3433lb # [ 18.321226] systemd[1]: Started DHCP Client.3434lb # [ 18.324480] systemd[1]: Reached target Network is Online.3435lb # [ 18.327918] systemd[1]: Started Envoy reverse proxy.3436lb # [ 18.336538] systemd[1]: Started mock-oidc.service.3437client # [ 18.424177] dhcpcd[528]: eth0: leased 10.0.2.15 for 86400 seconds3438client # [ 18.424549] dhcpcd[528]: eth0: adding route to 10.0.2.0/243439client # [ 18.424687] dhcpcd[528]: eth0: adding default route via 10.0.2.23440lb # [ 18.433465] mock-oidc-server[903]: Mock OIDC Server running3441lb # [ 18.434564] mock-oidc-server[903]: OIDC Address: 192.168.1.2:80803442lb # [ 18.435888] mock-oidc-server[903]: Issue Address: 0.0.0.0:80813443lb # [ 18.437431] mock-oidc-server[903]: Issuer: http://192.168.1.2:8080/oidc3444lb # [ 18.438884] mock-oidc-server[903]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3445lb # [ 18.440717] mock-oidc-server[903]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3446lb # [ 18.442278] mock-oidc-server[903]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3447client # [ 18.601955] systemd[1]: Started DHCP Client.3448client # [ 18.603185] systemd[1]: Reached target Multi-User System.3449client # [ 18.604796] systemd[1]: Startup finished in 935ms (kernel) + 4.570s (initrd) + 13.098s (userspace) = 18.604s.3450lb # [ 18.967261] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120)3451lb # [ 18.969377] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:469] statically linked extensions:3452lb # [ 18.971118] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3453lb # [ 18.974122] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router3454lb # [ 19.018622] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3455lb # [ 19.025733] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13456lb # [ 19.028290] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3457lb # [ 19.030622] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3458lb # [ 19.032969] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3459lb # [ 19.035376] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3460lb # [ 19.040492] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3461lb # [ 19.045837] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3462lb # [ 19.048467] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3463lb # [ 19.053783] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3464lb # [ 19.055633] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme3465lb # [ 19.059405] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input3466lb # [ 19.062261] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3467lb # [ 19.065152] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3468lb # [ 19.066822] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83469lb # [ 19.069185] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3470lb # [ 19.072294] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3471lb # [ 19.074143] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3472lb # [ 19.075794] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3473lb # [ 19.079181] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3474lb # [ 19.081343] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3475lb # [ 19.083710] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3476lb # [ 19.089958] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3477lb # [ 19.092494] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle3478lb # [ 19.094695] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3479lb # [ 19.097425] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3480lb # [ 19.102391] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3481lb # [ 19.106356] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3482lb # [ 19.112111] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3483lb # [ 19.117396] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3484lb # [ 19.119820] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3485lb # [ 19.125281] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3486lb # [ 19.129179] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3487lb # [ 19.131880] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3488lb # [ 19.135405] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3489lb # [ 19.143227] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3490lb # [ 19.145641] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3491lb # [ 19.147840] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3492lb # [ 19.150349] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3493lb # [ 19.152420] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3494lb # [ 19.154383] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3495lb # [ 19.158246] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3496lb # [ 19.161494] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23497lb # [ 19.163936] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3498lb # [ 19.167301] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3499lb # [ 19.173233] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23500lb # [ 19.175053] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3501lb # [ 19.181751] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3502lb # [ 19.183907] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query3503lb # [ 19.187531] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3504lb # [ 19.191474] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3505lb # [ 19.194489] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats3506lb # [ 19.197968] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3507lb # [ 19.201636] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3508lb # [ 19.203567] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3509lb # [ 19.206408] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3510lb # [ 19.209630] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override3511lb # [ 19.213183] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert3512lb # [ 19.215234] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3513lb # [ 19.220334] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory3514worker2 # [ 19.212940] dhcpcd[647]: eth0: leased 10.0.2.15 for 86400 seconds3515lb # [ 19.223866] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3516lb # [ 19.228564] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3517worker2 # [ 19.220682] dhcpcd[647]: eth0: adding route to 10.0.2.0/243518lb # [ 19.230975] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3519lb # [ 19.233353] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23520worker2 # [ 19.223243] dhcpcd[647]: eth0: adding default route via 10.0.2.23521lb # [ 19.235300] envoy[902]: [2026-09-20 09:33:15.590][902][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3522lb # [ 19.241965] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3523lb # [ 19.246733] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3524lb # [ 19.255656] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3525lb # [ 19.264457] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3526lb # [ 19.269304] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3527lb # [ 19.279431] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3528lb # [ 19.285981] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3529lb # [ 19.294116] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3530lb # [ 19.303707] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3531lb # [ 19.306373] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter3532lb # [ 19.313785] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3533lb # [ 19.332100] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3534lb # [ 19.334801] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3535lb # [ 19.341289] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3536lb # [ 19.348920] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3537lb # [ 19.352751] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit3538lb # [ 19.356910] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3539lb # [ 19.359828] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3540lb # [ 19.362974] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3541lb # [ 19.371742] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3542lb # [ 19.375611] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls3543worker1 # [ 19.427488] systemd[1]: Starting Nix worker-protocol over gRPC...3544worker2 # [ 19.380858] systemd[1]: Starting Nix worker-protocol over gRPC...3545lb # [ 19.381483] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3546lb # [ 19.400319] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3547lb # [ 19.405810] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3548lb # [ 19.419439] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3549lb # [ 19.434279] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3550lb # [ 19.439673] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3551lb # [ 19.447246] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3552lb # [ 19.450349] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3553lb # [ 19.453195] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3554lb # [ 19.455861] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3555lb # [ 19.458981] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3556lb # [ 19.461860] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3557lb # [ 19.463918] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3558lb # [ 19.466575] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3559worker2 # [ 19.461394] systemd[1]: Started DHCP Client.3560worker2 # [ 19.463639] systemd[1]: Reached target Multi-User System.3561lb # [ 19.469379] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd3562lb # [ 19.479573] envoy[902]: [2026-09-20 09:33:15.591][902][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3563lb # [ 19.483261] envoy[902]: [2026-09-20 09:33:15.638][902][info][main] [source/server/server.cc:531] HTTP header map info:3564lb # [ 19.485309] envoy[902]: [2026-09-20 09:33:15.640][902][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3565lb # [ 19.502298] envoy[902]: [2026-09-20 09:33:15.640][902][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3566lb # [ 19.504582] envoy[902]: [2026-09-20 09:33:15.640][902][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3567lb # [ 19.514402] envoy[902]: [2026-09-20 09:33:15.640][902][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3568lb # [ 19.518424] envoy[902]: [2026-09-20 09:33:15.830][902][info][main] [source/server/server.cc:968] runtime: {}3569lb # [ 19.521319] envoy[902]: [2026-09-20 09:33:15.844][902][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3570lb # [ 19.523508] envoy[902]: [2026-09-20 09:33:15.844][902][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013571lb # [ 19.525785] envoy[902]: [2026-09-20 09:33:15.845][902][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013572lb # [ 19.531867] envoy[902]: [2026-09-20 09:33:15.852][902][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3573lb # [ 19.540619] envoy[902]: [2026-09-20 09:33:15.852][902][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3574lb # [ 19.544675] envoy[902]: [2026-09-20 09:33:15.852][902][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3575lb # [ 19.549060] envoy[902]: [2026-09-20 09:33:16.020][902][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3576lb # [ 19.550730] envoy[902]: [2026-09-20 09:33:16.067][902][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3577lb # [ 19.553333] envoy[902]: [2026-09-20 09:33:16.069][902][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3578lb # [ 19.556994] envoy[902]: [2026-09-20 09:33:16.069][902][info][main] [source/server/server.cc:1077] starting main dispatch loop3579worker2 # [ 19.627680] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3580worker2 # [ 19.640246] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13581worker1 # [ 19.699712] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3582worker1 # [ 19.713561] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13583worker2 # [ 19.686431] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3584worker2 # [ 19.691432] systemd[1]: Started Nix worker-protocol over gRPC.3585worker2 # [ 19.693119] systemd[1]: Startup finished in 943ms (kernel) + 5.453s (initrd) + 13.291s (userspace) = 19.688s.3586worker1 # [ 19.761154] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3587worker1 # [ 19.767267] systemd[1]: Started Nix worker-protocol over gRPC.3588lb # [ 19.746035] envoy[902]: [2026-09-20 09:33:16.370][902][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3589lb # [ 19.760652] envoy[902]: [2026-09-20 09:33:16.375][902][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3590worker2 # [ 19.761625] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513591lb # [ 19.770548] envoy[902]: [2026-09-20 09:33:16.376][902][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3592lb # [ 19.778243] envoy[902]: [2026-09-20 09:33:16.376][902][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3593worker2 # [ 19.778454] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3594worker1 # [ 19.839615] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500513595worker1 # [ 19.841934] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3596worker2 # [ 19.979810] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513597worker2 # [ 19.985356] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3598worker1 # [ 20.043589] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500513599worker1 # [ 20.049271] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3600worker2 # [ 20.186291] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513601worker2 # [ 20.191924] nix-grpc-daemon[759]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3602worker1 # [ 20.250042] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500513603worker1 # [ 20.255590] nix-grpc-daemon[774]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3604worker2 # [ 20.393056] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513605worker2 # [ 20.399053] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3606worker1 # [ 20.456666] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513607worker1 # [ 20.462430] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3608worker2 # [ 20.600289] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513609worker2 # [ 20.605936] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3610worker1 # [ 20.663487] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513611worker1 # [ 20.669296] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3612worker2 # [ 20.808958] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513613worker2 # [ 20.813387] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3614worker1 # [ 20.873271] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513615worker1 # [ 20.877885] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3616worker2 # [ 21.009626] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513617worker2 # [ 21.015153] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3618worker1 # [ 21.073379] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513619worker1 # [ 21.079284] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3620worker2 # [ 21.216090] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513621worker2 # [ 21.221508] nix-grpc-daemon[759]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3622worker1 # [ 21.281283] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500513623worker1 # [ 21.286248] nix-grpc-daemon[774]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker2 # [ 21.422955] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513625worker2 # [ 21.428947] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3626worker1 # [ 21.487838] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513627worker1 # [ 21.493368] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker2 # [ 21.630522] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513629worker2 # [ 21.636427] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3630worker1 # [ 21.692364] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513631worker1 # [ 21.696685] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker2 # [ 21.840387] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513633worker1 # [ 21.895648] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513634worker2 # [ 21.844948] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3635worker1 # [ 21.900257] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker2 # [ 22.040305] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 22.097795] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513638worker2 # [ 22.045938] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker1 # [ 22.102683] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker2 # [ 22.249010] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 22.303642] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500513642worker2 # [ 22.253554] nix-grpc-daemon[759]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker1 # [ 22.309425] nix-grpc-daemon[774]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker2 # [ 22.450375] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513645worker2 # [ 22.456108] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3646worker1 # [ 22.511775] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513647worker1 # [ 22.518348] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker2 # [ 22.657198] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 22.712990] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513650worker2 # [ 22.663364] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker1 # [ 22.718664] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker2 # [ 22.866061] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 22.921668] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513654worker2 # [ 22.872402] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker1 # [ 22.925999] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 23.122662] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513657worker2 # [ 23.073772] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513658worker1 # [ 23.128484] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3659worker2 # [ 23.080429] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 23.330407] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500513661worker2 # [ 23.281105] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513662worker1 # [ 23.336555] nix-grpc-daemon[774]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3663worker2 # [ 23.286747] nix-grpc-daemon[759]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 23.538607] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513665worker2 # [ 23.488083] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513666worker2 # [ 23.493123] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3667worker1 # [ 23.544986] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker2 # [ 23.690247] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 23.746013] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513670worker2 # [ 23.697186] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3671worker1 # [ 23.751988] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker2 # [ 23.900404] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 23.954915] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513674worker2 # [ 23.905412] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker1 # [ 23.959337] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker2 # [ 24.101406] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 24.155284] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513678worker2 # [ 24.106951] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker1 # [ 24.161357] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker2 # [ 24.308209] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 24.362220] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500513682worker2 # [ 24.313648] nix-grpc-daemon[759]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker1 # [ 24.367999] nix-grpc-daemon[774]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker2 # [ 24.514745] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 24.569292] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513686worker2 # [ 24.520818] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3687worker1 # [ 24.574904] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker2 # [ 24.722481] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 24.775890] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513690worker2 # [ 24.728529] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker1 # [ 24.781622] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker2 # [ 24.932456] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 24.985114] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513694worker2 # [ 24.936909] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker1 # [ 24.989525] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker2 # [ 25.133208] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 25.185751] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500513698worker2 # [ 25.139280] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker1 # [ 25.191423] nix-grpc-daemon[774]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker2 # [ 25.340297] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 25.392980] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513702worker2 # [ 25.346109] nix-grpc-daemon[759]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3703worker1 # [ 25.398515] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 25.595841] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513705worker2 # [ 25.547333] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513706worker1 # [ 25.602070] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3707worker2 # [ 25.553040] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 25.804851] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513709worker2 # [ 25.753751] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513710worker1 # [ 25.809416] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3711worker2 # [ 25.759427] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 26.007793] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513713worker2 # [ 25.961318] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513714worker1 # [ 26.011269] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3715worker2 # [ 25.963627] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 26.209150] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500513717worker2 # [ 26.162356] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513718worker1 # [ 26.214782] nix-grpc-daemon[774]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3719worker2 # [ 26.168124] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 26.415997] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513721worker2 # [ 26.369271] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513722worker1 # [ 26.421649] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3723worker2 # [ 26.374466] nix-grpc-daemon[759]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 26.622298] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513725worker2 # [ 26.571772] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513726worker1 # [ 26.627880] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker2 # [ 26.577633] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 26.829005] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513729worker2 # [ 26.781029] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513730worker1 # [ 26.837049] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker2 # [ 26.786814] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker2 # [ 26.984738] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 27.040914] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513734worker2 # [ 26.989154] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker1 # [ 27.045145] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker2 # [ 27.185725] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 27.240421] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500513738worker2 # [ 27.191319] nix-grpc-daemon[759]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3739worker1 # [ 27.245957] nix-grpc-daemon[774]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker2 # [ 27.392569] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 27.446996] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513742worker2 # [ 27.398422] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3743worker1 # [ 27.452660] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker2 # [ 27.600794] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 27.653484] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513746worker2 # [ 27.606992] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3747worker1 # [ 27.658870] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker2 # [ 27.804087] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 27.859842] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513750worker2 # [ 27.810195] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker1 # [ 27.866062] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker2 # [ 28.014478] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 28.068972] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513754worker2 # [ 28.019295] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3755worker1 # [ 28.075444] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756lb # [ 28.118505] rustfs-bucket-start[995]: mb s3://farm3757lb # [ 28.128331] systemd[1]: Finished rustfs-bucket.service.3758lb # [ 28.141716] systemd[1]: Starting niks3 server...3759worker2 # [ 28.214579] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513760worker2 # [ 28.220434] nix-grpc-daemon[759]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3761worker1 # [ 28.277285] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500513762worker1 # [ 28.283740] nix-grpc-daemon[774]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3763lb # [ 28.372283] postgres[1011]: [1011] ERROR: relation "goose_db_version" does not exist at character 363764lb # [ 28.376184] postgres[1011]: [1011] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3765lb # [ 28.411625] niks3-server[1004]: 2026/09/20 09:33:25 OK 20241026095416_initial_model.sql (22.5ms)3766lb # [ 28.415112] niks3-server[1004]: 2026/09/20 09:33:25 OK 20251210153512_drop_unused_gin_index.sql (3.86ms)3767lb # [ 28.420893] niks3-server[1004]: 2026/09/20 09:33:25 OK 20251218171726_add_pins.sql (5.71ms)3768lb # [ 28.425722] niks3-server[1004]: 2026/09/20 09:33:25 OK 20260628120000_add_object_size_and_stats.sql (4.79ms)3769lb # [ 28.431179] niks3-server[1004]: 2026/09/20 09:33:25 OK 20260905000000_add_claims.sql (5.42ms)3770worker2 # [ 28.421538] nix-grpc-daemon[759]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500513771lb # [ 28.436884] niks3-server[1004]: 2026/09/20 09:33:25 OK 20260920000000_drop_claims.sql (5.57ms)3772lb # [ 28.438897] niks3-server[1004]: 2026/09/20 09:33:25 goose: successfully migrated database to version: 202609200000003773worker2 # [ 28.427390] nix-grpc-daemon[759]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774lb # [ 28.444387] niks3-server[1004]: 2026/09/20 09:33:25 OK 1_commit_pending_closure.sql (7.48ms)3775worker1 # [ 28.484579] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513776lb # [ 28.447081] niks3-server[1004]: 2026/09/20 09:33:25 OK 2_object_stats_trigger.sql (2.56ms)3777lb # [ 28.448772] niks3-server[1004]: 2026/09/20 09:33:25 goose: up to current file version: 23778worker1 # [ 28.490169] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3779lb # [ 28.452667] niks3-server[1004]: 2026/09/20 09:33:25 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3780lb # [ 28.499688] niks3-server[1004]: 2026/09/20 09:33:25 INFO Read proxy enabled — serving cache objects from S33781lb # [ 28.501742] niks3-server[1004]: 2026/09/20 09:33:25 INFO Using socket-activated listener address=0.0.0.0:57513782lb # [ 28.506457] niks3-server[1004]: 2026/09/20 09:33:25 INFO systemd watchdog enabled interval=15s3783lb # [ 28.507742] systemd[1]: Started niks3 server.3784lb # [ 28.510324] systemd[1]: Reached target Multi-User System.3785lb # [ 28.513844] niks3-server[1004]: 2026/09/20 09:33:25 INFO Starting HTTP server address=0.0.0.0:57513786lb # [ 28.515109] systemd[1]: Startup finished in 944ms (kernel) + 5.466s (initrd) + 22.099s (userspace) = 28.509s.3787lb # [ 28.520357] niks3-server[1004]: 2026/09/20 09:33:25 INFO lead: acquired remote=192.168.1.3:431703788worker1 # [ 28.564829] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=scheduler_take_over3789worker2 # [ 28.628196] nix-grpc-daemon[759]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500513790worker2 # [ 28.633812] nix-grpc-daemon[759]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3791lb: (finished: waiting for unit niks3.service, in 15.09 seconds)3792lb: waiting for TCP port 5751 on localhost3793worker1 # [ 28.691215] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513794worker1 # [ 28.696337] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3795lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3796lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)3797lb: waiting for unit envoy.service3798lb: (finished: waiting for unit envoy.service, in 0.06 seconds)3799lb: waiting for TCP port 50051 on localhost3800lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3801lb # [ 28.829329] envoy[902]: conn peer=[::1]:34730 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33802lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3803worker2 # [ 28.835680] nix-grpc-daemon[759]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500513804worker1 # [ 28.896282] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500513805worker1 # [ 28.905187] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03806worker1 # [ 28.914102] nix-grpc-daemon[774]: ts=2026-09-20T09:33:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03807lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3808??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.3809 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1283810lb: must succeed: curl -sf localhost:9901/clusters3811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.15 seconds)3812lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.15 seconds)3813subtest: one node schedules and both builders hold a WorkerSession on it3814lb: must succeed: curl -sf localhost:9901/clusters3815lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3816worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true3817worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.12 seconds)3818(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.19 seconds)3819subtest: balancer auth: client cert, bearer token, nothing, unknown CN3820worker1 # [ 29.850837] nix-grpc-daemon[774]: ts=2026-09-20T09:33:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443443821worker1 # [ 29.874697] systemd[1]: Started Nix Daemon.3822worker1 # [ 30.252072] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)3823worker1 # [ 30.296832] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)3824worker1 # [ 30.370166] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)3825worker1 # [ 30.573978] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5)3826worker1 # [ 30.971614] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 690 ms (attempt 4/5)3827worker1 # [ 31.664390] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3828worker2 # [ 31.666874] nix-grpc-daemon[759]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364763829worker2 # [ 31.679592] systemd[1]: Started Nix Daemon.3830worker2 # [ 32.623081] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3831worker2 # [ 32.658317] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)3832worker2 # [ 32.719775] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5)3833worker2 # [ 32.890612] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 3/5)3834worker2 # [ 32.957283] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 795 ms (attempt 4/5)3835worker2 # [ 33.765037] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3836worker2 # [ 33.781913] nix-grpc-daemon[759]: ts=2026-09-20T09:33:30Z level=debug event=path_info_miss path=5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test error="error: path '/nix/store/5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test' is required, but there is no substituter that can build it"3837worker2 # [ 33.797975] nix-grpc-daemon[759]: ts=2026-09-20T09:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364763838worker2 # [ 33.806205] nix-grpc-daemon[759]: ts=2026-09-20T09:33:30Z level=debug event=path_info_miss path=5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test error="error: path '/nix/store/5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test' is required, but there is no substituter that can build it"3839client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3840client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.12 seconds)3841client: must succeed: install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3842client: (finished: must succeed: install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.06 seconds)3843worker1 # [ 34.327741] nix-grpc-daemon[774]: ts=2026-09-20T09:33:29Z level=info event=oidc_jwks provider=mock keys=13844worker1 # [ 34.333746] nix-grpc-daemon[774]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:443443845worker1 # [ 34.345594] nix-grpc-daemon[774]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:443443846worker1 # [ 34.360143] nix-grpc-daemon[774]: ts=2026-09-20T09:33:29Z level=debug event=path_info_miss path=5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test error="error: path '/nix/store/5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test' is required, but there is no substituter that can build it"3847worker1 # [ 34.374459] nix-grpc-daemon[774]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:443443848worker1 # [ 34.382493] nix-grpc-daemon[774]: ts=2026-09-20T09:33:29Z level=debug event=path_info_miss path=5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test error="error: path '/nix/store/5rrxzpsp33jzaxb9v245gd67mmn0zxp0-nixos-system-client-test' is required, but there is no substituter that can build it"3849client: must succeed: rm -r /var/lib/nix-grpc-store3850client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3851worker2 # [ 34.621285] nix-grpc-daemon[759]: ts=2026-09-20T09:33:31Z level=info event=denied method=QueryMissing cn=- role=none3852worker2 # [ 34.861282] nix-grpc-daemon[759]: ts=2026-09-20T09:33:31Z level=info event=denied method=QueryMissing cn=stranger role=none3853(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.48 seconds)3854subtest: DAG build is scheduled across workers and lands in the cache3855client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23856worker1 # [ 35.198045] nix-grpc-daemon[774]: ts=2026-09-20T09:33:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443443857client # don't know how to build these paths:3858client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3859worker1 # [ 35.213188] nix-grpc-daemon[774]: ts=2026-09-20T09:33:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443443860worker1 # [ 35.248140] nix-grpc-daemon[774]: ts=2026-09-20T09:33:30Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03861worker1 # [ 35.253951] nix-grpc-daemon[774]: ts=2026-09-20T09:33:30Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13862worker2 # [ 35.242134] nix-grpc-daemon[759]: ts=2026-09-20T09:33:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364763863worker2 # [ 35.273941] nix-grpc-daemon[759]: ts=2026-09-20T09:33:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364763864client # copying 1 paths to 'lb:50051'...3865worker2 # [ 35.290319] nix-grpc-daemon[759]: ts=2026-09-20T09:33:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364763866client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3867worker2 # [ 35.317111] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3868worker2 # [ 35.353271] nix-grpc-daemon[759]: ts=2026-09-20T09:33:31Z level=info event=niks3_push_started pid=8423869worker2 # [ 35.829883] nix-daemon[832]: accepted connection from pid 855, user nix-grpc-daemon (trusted)3870worker2 # [ 35.857673] nix-daemon[832]: reaped child process 858, status = succeeded3871worker2 # [ 36.052805] nix-daemon[832]: accepted connection from pid 860, user nix-grpc-daemon (trusted)3872worker2 # [ 36.072002] nix-daemon[832]: reaped child process 864, status = succeeded3873lb # [ 36.094654] niks3-server[1004]: 2026/09/20 09:33:32 INFO Received uploads request method=POST path=/api/pending_closures3874worker2 # [ 36.112980] nix-grpc-daemon[842]: time=2026-09-20T09:33:32.738Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3875worker2 # [ 36.114606] nix-grpc-daemon[842]: time=2026-09-20T09:33:32.738Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3876lb # [ 36.163591] niks3-server[1004]: 2026/09/20 09:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3877lb # [ 36.192108] niks3-server[1004]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3878lb # [ 36.209048] niks3-server[1004]: 2026/09/20 09:33:32 INFO Signed narinfos id=1 count=13879worker2 # [ 36.202813] nix-grpc-daemon[842]: time=2026-09-20T09:33:32.829Z level=INFO msg="Uploading 1 narinfos"3880lb # [ 36.226879] niks3-server[1004]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3881lb # [ 36.236380] niks3-server[1004]: 2026/09/20 09:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3882lb # [ 36.243906] niks3-server[1004]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3883lb # [ 36.251838] niks3-server[1004]: 2026/09/20 09:33:32 INFO Completed upload id=13884worker2 # [ 36.242237] nix-grpc-daemon[842]: time=2026-09-20T09:33:32.870Z level=INFO msg="Upload complete. (601ms)"3885worker2 # [ 36.257218] nix-grpc-daemon[759]: ts=2026-09-20T09:33:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=5043886worker2 # [ 36.269093] nix-grpc-daemon[759]: ts=2026-09-20T09:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364763887worker2 # [ 36.285104] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3888worker2 # [ 36.288521] nix-daemon[832]: reaped child process 840, status = succeeded3889client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3890client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3891worker2 # [ 36.303253] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3892client # farm-a> LOG-farm-a-t13893worker2 # [ 37.125487] nix-daemon[832]: reaped child process 868, status = succeeded3894worker2 # [ 37.287631] nix-daemon[832]: accepted connection from pid 883, user nix-grpc-daemon (trusted)3895worker2 # [ 37.305078] nix-daemon[832]: reaped child process 886, status = succeeded3896lb # [ 37.327835] niks3-server[1004]: 2026/09/20 09:33:33 INFO Received uploads request method=POST path=/api/pending_closures3897worker2 # [ 37.331376] nix-grpc-daemon[842]: time=2026-09-20T09:33:33.958Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3898worker2 # [ 37.333430] nix-grpc-daemon[842]: time=2026-09-20T09:33:33.958Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3899lb # [ 37.379016] niks3-server[1004]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3900lb # [ 37.394535] niks3-server[1004]: 2026/09/20 09:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3901worker2 # [ 37.390049] nix-grpc-daemon[842]: time=2026-09-20T09:33:34.016Z level=INFO msg="Uploading 1 narinfos"3902lb # [ 37.407675] niks3-server[1004]: 2026/09/20 09:33:34 INFO Signed narinfos id=2 count=13903lb # [ 37.412518] niks3-server[1004]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3904lb # [ 37.415386] niks3-server[1004]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3905lb # [ 37.420988] niks3-server[1004]: 2026/09/20 09:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3906lb # [ 37.430541] niks3-server[1004]: 2026/09/20 09:33:34 INFO Completed upload id=23907lb # [ 37.432827] niks3-server[1004]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3908worker2 # [ 37.422555] nix-grpc-daemon[842]: time=2026-09-20T09:33:34.050Z level=INFO msg="Upload complete. (308ms)"3909worker2 # [ 37.432962] nix-grpc-daemon[759]: ts=2026-09-20T09:33:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13910worker2 # [ 37.448656] nix-daemon[832]: reaped child process 866, status = succeeded3911worker1 # [ 37.496720] nix-grpc-daemon[774]: ts=2026-09-20T09:33:33Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03912worker1 # [ 37.497026] nix-grpc-daemon[774]: ts=2026-09-20T09:33:33Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=13913worker2 # [ 37.490414] nix-grpc-daemon[759]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364763914worker2 # [ 37.506769] nix-grpc-daemon[759]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364763915worker2 # [ 37.521922] nix-grpc-daemon[842]: time=2026-09-20T09:33:34.149Z level=INFO msg="All 1 paths already cached"3916client # copying 1 paths to 'lb:50051'...3917worker2 # [ 37.532357] nix-grpc-daemon[759]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364763918client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3919worker2 # [ 37.551621] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3920worker2 # [ 37.773340] nix-daemon[832]: accepted connection from pid 890, user nix-grpc-daemon (trusted)3921worker2 # [ 37.794405] nix-daemon[832]: reaped child process 893, status = succeeded3922worker2 # [ 37.967108] nix-daemon[832]: accepted connection from pid 895, user nix-grpc-daemon (trusted)3923worker2 # [ 37.988476] nix-daemon[832]: reaped child process 898, status = succeeded3924lb # [ 38.009175] niks3-server[1004]: 2026/09/20 09:33:34 INFO Received uploads request method=POST path=/api/pending_closures3925worker2 # [ 38.009520] nix-grpc-daemon[842]: time=2026-09-20T09:33:34.637Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3926worker2 # [ 38.011017] nix-grpc-daemon[842]: time=2026-09-20T09:33:34.637Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3927lb # [ 38.039164] niks3-server[1004]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3928worker2 # [ 38.044988] nix-grpc-daemon[842]: time=2026-09-20T09:33:34.672Z level=INFO msg="Uploading 1 narinfos"3929lb # [ 38.056637] niks3-server[1004]: 2026/09/20 09:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3930lb # [ 38.066902] niks3-server[1004]: 2026/09/20 09:33:34 INFO Signed narinfos id=3 count=13931worker2 # [ 38.067076] nix-grpc-daemon[842]: time=2026-09-20T09:33:34.695Z level=INFO msg="Upload complete. (494ms)"3932lb # [ 38.071675] niks3-server[1004]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3933lb # [ 38.084900] niks3-server[1004]: 2026/09/20 09:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3934worker2 # [ 38.075716] nix-grpc-daemon[759]: ts=2026-09-20T09:33:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=6243935lb # [ 38.089907] niks3-server[1004]: 2026/09/20 09:33:34 INFO Completed upload id=33936lb # [ 38.092459] niks3-server[1004]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3937worker2 # [ 38.086260] nix-grpc-daemon[759]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364763938worker2 # [ 38.092912] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3939worker2 # [ 38.097674] nix-daemon[832]: reaped child process 888, status = succeeded3940client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3941client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3942worker2 # [ 38.110725] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3943client # farm-b> LOG-farm-b-t13944worker2 # [ 39.334780] nix-daemon[832]: reaped child process 902, status = succeeded3945lb # [ 39.400623] envoy[902]: conn peer=[2001:db8:1::1]:33210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=40563946lb # [ 39.405719] envoy[902]: conn peer=[2001:db8:1::1]:39164 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=863947lb # [ 39.410280] envoy[902]: conn peer=[2001:db8:1::1]:39176 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=123948lb # [ 39.414877] envoy[902]: conn peer=[2001:db8:1::1]:39186 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=183949worker2 # [ 39.496513] nix-daemon[832]: accepted connection from pid 918, user nix-grpc-daemon (trusted)3950worker2 # [ 39.514451] nix-daemon[832]: reaped child process 921, status = succeeded3951lb # [ 39.536924] niks3-server[1004]: 2026/09/20 09:33:36 INFO Received uploads request method=POST path=/api/pending_closures3952worker2 # [ 39.539013] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.166Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3953worker2 # [ 39.540592] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.166Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"3954lb # [ 39.575647] niks3-server[1004]: 2026/09/20 09:33:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign3955worker2 # [ 39.572104] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.199Z level=INFO msg="Uploading 1 narinfos"3956lb # [ 39.592849] niks3-server[1004]: 2026/09/20 09:33:36 INFO Signed narinfos id=4 count=13957lb # [ 39.605607] niks3-server[1004]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst3958lb # [ 39.613041] niks3-server[1004]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls3959lb # [ 39.621257] niks3-server[1004]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3960worker2 # [ 39.615787] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.243Z level=INFO msg="Upload complete. (295ms)"3961lb # [ 39.626374] niks3-server[1004]: 2026/09/20 09:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete3962lb # [ 39.630620] niks3-server[1004]: 2026/09/20 09:33:36 INFO Completed upload id=43963lb # [ 39.633805] niks3-server[1004]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo3964worker2 # [ 39.627698] nix-grpc-daemon[759]: ts=2026-09-20T09:33:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=13965worker1 # [ 39.688487] nix-grpc-daemon[774]: ts=2026-09-20T09:33:35Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=03966worker2 # [ 39.643931] nix-daemon[832]: reaped child process 900, status = succeeded3967worker1 # [ 39.688937] nix-grpc-daemon[774]: ts=2026-09-20T09:33:35Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=13968worker2 # [ 39.682363] nix-grpc-daemon[759]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364763969worker2 # [ 39.697688] nix-grpc-daemon[759]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364763970client # copying 1 paths to 'lb:50051'...3971worker2 # [ 39.712932] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.340Z level=INFO msg="All 2 paths already cached"3972worker2 # [ 39.718272] nix-grpc-daemon[759]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364763973client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...3974worker2 # [ 39.741452] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3975worker2 # [ 39.939561] nix-daemon[832]: accepted connection from pid 925, user nix-grpc-daemon (trusted)3976worker2 # [ 39.960127] nix-daemon[832]: reaped child process 928, status = succeeded3977worker2 # [ 40.129598] nix-daemon[832]: accepted connection from pid 930, user nix-grpc-daemon (trusted)3978worker2 # [ 40.147220] nix-daemon[832]: reaped child process 933, status = succeeded3979lb # [ 40.168169] niks3-server[1004]: 2026/09/20 09:33:36 INFO Received uploads request method=POST path=/api/pending_closures3980worker2 # [ 40.168163] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.795Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"3981worker2 # [ 40.169615] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.795Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"3982lb # [ 40.196562] niks3-server[1004]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst3983worker2 # [ 40.205745] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.833Z level=INFO msg="Uploading 1 narinfos"3984lb # [ 40.208856] niks3-server[1004]: 2026/09/20 09:33:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign3985lb # [ 40.221476] niks3-server[1004]: 2026/09/20 09:33:36 INFO Signed narinfos id=5 count=13986lb # [ 40.227511] niks3-server[1004]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls3987lb # [ 40.235787] niks3-server[1004]: 2026/09/20 09:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete3988lb # [ 40.242695] niks3-server[1004]: 2026/09/20 09:33:36 INFO Completed upload id=53989worker2 # [ 40.232834] nix-grpc-daemon[842]: time=2026-09-20T09:33:36.860Z level=INFO msg="Upload complete. (473ms)"3990lb # [ 40.245053] niks3-server[1004]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo3991worker2 # [ 40.240265] nix-grpc-daemon[759]: ts=2026-09-20T09:33:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=7523992worker2 # [ 40.254735] nix-grpc-daemon[759]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364763993worker2 # [ 40.261080] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3994worker2 # [ 40.265844] nix-daemon[832]: reaped child process 923, status = succeeded3995client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...3996client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3997worker2 # [ 40.279034] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)3998client # farm-top> LOG-farm-top-t13999worker2 # [ 40.908436] nix-daemon[832]: reaped child process 937, status = succeeded4000worker2 # [ 41.072922] nix-daemon[832]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4001worker2 # [ 41.090491] nix-daemon[832]: reaped child process 955, status = succeeded4002lb # [ 41.114795] niks3-server[1004]: 2026/09/20 09:33:37 INFO Received uploads request method=POST path=/api/pending_closures4003worker2 # [ 41.118110] nix-grpc-daemon[842]: time=2026-09-20T09:33:37.744Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4004worker2 # [ 41.119693] nix-grpc-daemon[842]: time=2026-09-20T09:33:37.744Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4005lb # [ 41.152300] niks3-server[1004]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4006lb # [ 41.168879] niks3-server[1004]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4007lb # [ 41.189500] niks3-server[1004]: 2026/09/20 09:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4008worker2 # [ 41.185229] nix-grpc-daemon[842]: time=2026-09-20T09:33:37.812Z level=INFO msg="Uploading 1 narinfos"4009lb # [ 41.199693] niks3-server[1004]: 2026/09/20 09:33:37 INFO Signed narinfos id=6 count=14010lb # [ 41.206500] niks3-server[1004]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4011lb # [ 41.217766] niks3-server[1004]: 2026/09/20 09:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4012worker2 # [ 41.211962] nix-grpc-daemon[842]: time=2026-09-20T09:33:37.840Z level=INFO msg="Upload complete. (323ms)"4013lb # [ 41.224811] niks3-server[1004]: 2026/09/20 09:33:37 INFO Completed upload id=64014lb # [ 41.226961] niks3-server[1004]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4015worker2 # [ 41.225085] nix-grpc-daemon[759]: ts=2026-09-20T09:33:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14016worker2 # [ 41.242695] nix-daemon[832]: reaped child process 935, status = succeeded4017client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 6.39 seconds)4018client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4019client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4020client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4021client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.04 seconds)4022client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4023client # copying 3 paths...4024client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4025client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4026client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4027client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4028(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.08 seconds)4029subtest: repeat is answered Cached by the scheduler without building4030worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4031worker1 # finding garbage collector roots...4032worker1 # deleting specified paths...4033worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4034worker1 # deleting unused links...4035worker1 # note: hard linking is currently saving 0.0 KiB4036worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4037worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4038worker2 # finding garbage collector roots...4039worker2 # deleting specified paths...4040worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4041worker2 # deleting unused links...4042worker2 # note: hard linking is currently saving 0.0 KiB4043worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.33 seconds)4044client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24045client # don't know how to build these paths:4046client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4047worker1 # [ 42.893398] nix-grpc-daemon[774]: ts=2026-09-20T09:33:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443444048worker2 # [ 42.845894] nix-grpc-daemon[759]: ts=2026-09-20T09:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764049worker2 # [ 42.857184] nix-grpc-daemon[842]: time=2026-09-20T09:33:39.484Z level=INFO msg="All 2 paths already cached"4050worker1 # [ 42.925594] nix-grpc-daemon[774]: ts=2026-09-20T09:33:38Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14051client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.31 seconds)4052client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4053client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4054lb: must succeed: curl -sf localhost:9901/clusters4055lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4056lb: must succeed: curl -sf localhost:9901/clusters4057lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4058worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4059worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.08 seconds)4060(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.46 seconds)4061subtest: inputs already in the cache are not sent to the scheduler4062lb: must succeed: curl -sf localhost:9901/clusters4063lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4064lb: must succeed: curl -sf localhost:9901/clusters4065lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4066worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4067worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4068client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24069worker2 # [ 43.866328] nix-grpc-daemon[759]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364764070client # don't know how to build these paths:4071client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4072worker1 # [ 43.930150] nix-grpc-daemon[774]: ts=2026-09-20T09:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443444073worker1 # [ 43.949405] nix-grpc-daemon[774]: ts=2026-09-20T09:33:39Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04074worker1 # [ 43.955460] nix-grpc-daemon[774]: ts=2026-09-20T09:33:39Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14075worker2 # [ 43.946053] nix-grpc-daemon[759]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764076worker2 # [ 43.961184] nix-grpc-daemon[759]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764077client # copying 1 paths to 'lb:50051'...4078worker2 # [ 43.977321] nix-grpc-daemon[842]: time=2026-09-20T09:33:40.603Z level=INFO msg="All 2 paths already cached"4079client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4080worker2 # [ 43.982619] nix-grpc-daemon[759]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764081worker2 # [ 43.994462] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4082worker2 # [ 44.205657] nix-daemon[832]: accepted connection from pid 973, user nix-grpc-daemon (trusted)4083worker2 # [ 44.225659] nix-daemon[832]: reaped child process 976, status = succeeded4084worker2 # [ 44.403052] nix-daemon[832]: accepted connection from pid 978, user nix-grpc-daemon (trusted)4085worker2 # [ 44.423217] nix-daemon[832]: reaped child process 981, status = succeeded4086lb # [ 44.446245] niks3-server[1004]: 2026/09/20 09:33:41 INFO Received uploads request method=POST path=/api/pending_closures4087worker2 # [ 44.445858] nix-grpc-daemon[842]: time=2026-09-20T09:33:41.073Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4088worker2 # [ 44.447340] nix-grpc-daemon[842]: time=2026-09-20T09:33:41.073Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4089lb # [ 44.471749] niks3-server[1004]: 2026/09/20 09:33:41 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4090lb # [ 44.487215] niks3-server[1004]: 2026/09/20 09:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4091worker2 # [ 44.483309] nix-grpc-daemon[842]: time=2026-09-20T09:33:41.111Z level=INFO msg="Uploading 1 narinfos"4092lb # [ 44.499379] niks3-server[1004]: 2026/09/20 09:33:41 INFO Signed narinfos id=7 count=14093lb # [ 44.505107] niks3-server[1004]: 2026/09/20 09:33:41 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4094lb # [ 44.513914] niks3-server[1004]: 2026/09/20 09:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4095worker2 # [ 44.508142] nix-grpc-daemon[842]: time=2026-09-20T09:33:41.136Z level=INFO msg="Upload complete. (491ms)"4096lb # [ 44.518872] niks3-server[1004]: 2026/09/20 09:33:41 INFO Completed upload id=74097lb # [ 44.522629] niks3-server[1004]: 2026/09/20 09:33:41 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4098worker2 # [ 44.516147] nix-grpc-daemon[759]: ts=2026-09-20T09:33:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=7524099worker2 # [ 44.528506] nix-grpc-daemon[759]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764100worker2 # [ 44.534761] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4101worker2 # [ 44.539682] nix-daemon[832]: reaped child process 971, status = succeeded4102client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4103client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4104worker2 # [ 44.553976] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4105client # farm-top> LOG-farm-top-t1b4106worker2 # [ 45.993525] nix-daemon[832]: reaped child process 985, status = succeeded4107worker2 # [ 46.165374] nix-daemon[832]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4108worker2 # [ 46.182987] nix-daemon[832]: reaped child process 1004, status = succeeded4109lb # [ 46.205649] niks3-server[1004]: 2026/09/20 09:33:42 INFO Received uploads request method=POST path=/api/pending_closures4110worker2 # [ 46.208398] nix-grpc-daemon[842]: time=2026-09-20T09:33:42.833Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4111worker2 # [ 46.209855] nix-grpc-daemon[842]: time=2026-09-20T09:33:42.834Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4112lb # [ 46.242790] niks3-server[1004]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4113lb # [ 46.259287] niks3-server[1004]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4114lb # [ 46.268831] niks3-server[1004]: 2026/09/20 09:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4115worker2 # [ 46.265057] nix-grpc-daemon[842]: time=2026-09-20T09:33:42.892Z level=INFO msg="Uploading 1 narinfos"4116lb # [ 46.274910] niks3-server[1004]: 2026/09/20 09:33:42 INFO Signed narinfos id=8 count=14117lb # [ 46.279043] niks3-server[1004]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4118lb # [ 46.288476] niks3-server[1004]: 2026/09/20 09:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4119lb # [ 46.297928] niks3-server[1004]: 2026/09/20 09:33:42 INFO Completed upload id=84120worker2 # [ 46.288091] nix-grpc-daemon[842]: time=2026-09-20T09:33:42.916Z level=INFO msg="Upload complete. (323ms)"4121lb # [ 46.300980] niks3-server[1004]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4122worker2 # [ 46.301090] nix-grpc-daemon[759]: ts=2026-09-20T09:33:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14123worker2 # [ 46.315948] nix-daemon[832]: reaped child process 983, status = succeeded4124client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.72 seconds)4125client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4126client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.24 seconds)4127lb: must succeed: curl -sf localhost:9901/clusters4128lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4129lb: must succeed: curl -sf localhost:9901/clusters4130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4131worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4132worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4133(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.37 seconds)4134subtest: two clients wanting the same drv build it once4135client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4136client # [ 46.950420] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4137client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4138client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4139client # Running as unit: dedup1.service; invocation ID: 0fdee0409bcc449692bcb60b3594bad64140client # [ 47.204996] nix[872]: don't know how to build these paths:4141client # [ 47.206853] nix[872]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4142worker2 # [ 47.128248] nix-grpc-daemon[759]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364764143worker1 # [ 47.196209] nix-grpc-daemon[774]: ts=2026-09-20T09:33:42Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04144worker1 # [ 47.196577] nix-grpc-daemon[774]: ts=2026-09-20T09:33:42Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14145worker1 # [ 47.241822] nix-grpc-daemon[774]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443444146worker1 # [ 47.262618] nix-grpc-daemon[774]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443444147client # [ 47.300515] nix[872]: copying 1 paths to 'lb:50051'...4148worker1 # [ 47.278034] nix-grpc-daemon[774]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443444149client # [ 47.304982] nix[872]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4150worker1 # [ 47.293442] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4151worker1 # [ 47.321937] nix-grpc-daemon[774]: ts=2026-09-20T09:33:42Z level=info event=niks3_push_started pid=8904152worker1 # [ 47.768536] nix-daemon[836]: accepted connection from pid 903, user nix-grpc-daemon (trusted)4153worker1 # [ 47.789702] nix-daemon[836]: reaped child process 906, status = succeeded4154worker1 # [ 47.964477] nix-daemon[836]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4155worker1 # [ 47.982730] nix-daemon[836]: reaped child process 911, status = succeeded4156lb # [ 47.954129] niks3-server[1004]: 2026/09/20 09:33:44 INFO Received uploads request method=POST path=/api/pending_closures4157worker1 # [ 48.007706] nix-grpc-daemon[890]: time=2026-09-20T09:33:43.623Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4158worker1 # [ 48.009638] nix-grpc-daemon[890]: time=2026-09-20T09:33:43.624Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4159lb # [ 47.989079] niks3-server[1004]: 2026/09/20 09:33:44 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4160worker1 # [ 48.043719] nix-grpc-daemon[890]: time=2026-09-20T09:33:43.661Z level=INFO msg="Uploading 1 narinfos"4161lb # [ 48.001882] niks3-server[1004]: 2026/09/20 09:33:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4162lb # [ 48.014710] niks3-server[1004]: 2026/09/20 09:33:44 INFO Signed narinfos id=9 count=14163client # [ 48.073305] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4164lb # [ 48.022259] niks3-server[1004]: 2026/09/20 09:33:44 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4165lb # [ 48.034485] niks3-server[1004]: 2026/09/20 09:33:44 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4166worker1 # [ 48.080257] nix-grpc-daemon[890]: time=2026-09-20T09:33:43.698Z level=INFO msg="Upload complete. (492ms)"4167lb # [ 48.039373] niks3-server[1004]: 2026/09/20 09:33:44 INFO Completed upload id=94168client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4169lb # [ 48.042345] niks3-server[1004]: 2026/09/20 09:33:44 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4170worker1 # [ 48.088139] nix-grpc-daemon[774]: ts=2026-09-20T09:33:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=0 paths=1 nar_bytes_in=4324171worker1 # [ 48.104751] nix-grpc-daemon[774]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443444172worker1 # [ 48.118728] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4173worker1 # [ 48.124562] nix-daemon[836]: reaped child process 888, status = succeeded4174client # Running as unit: dedup2.service; invocation ID: 4b36d3d9cf9042ae8a38bcccfdc201e04175client # [ 48.161395] nix[872]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4176worker1 # [ 48.142027] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4177client # [ 48.410754] nix[887]: don't know how to build these paths:4178client # [ 48.412699] nix[887]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4179worker2 # [ 48.335058] nix-grpc-daemon[759]: ts=2026-09-20T09:33:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364764180worker1 # [ 48.401198] nix-grpc-daemon[774]: ts=2026-09-20T09:33:44Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04181worker1 # [ 48.416089] nix-grpc-daemon[774]: ts=2026-09-20T09:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443444182client # [ 48.446900] nix[887]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4183client: must succeed: sleep 24184lb # [ 49.400209] envoy[902]: conn peer=[2001:db8:1::1]:39198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9519 tx=3806 ms=61104185lb # [ 49.400585] envoy[902]: conn peer=[2001:db8:1::1]:56620 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=554186lb # [ 49.400698] envoy[902]: conn peer=[2001:db8:1::1]:56626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1727 ms=24464187client: (finished: must succeed: sleep 2, in 2.06 seconds)4188??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4189 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394190client: waiting for success: ! systemctl is-active dedup1 dedup24191??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4192 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394193worker1 # [ 51.796418] nix-daemon[836]: reaped child process 919, status = succeeded4194worker1 # [ 51.976482] nix-daemon[836]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4195worker1 # [ 51.993729] nix-daemon[836]: reaped child process 949, status = succeeded4196lb # [ 51.965988] niks3-server[1004]: 2026/09/20 09:33:48 INFO Received uploads request method=POST path=/api/pending_closures4197worker1 # [ 52.018583] nix-grpc-daemon[890]: time=2026-09-20T09:33:47.635Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4198worker1 # [ 52.021198] nix-grpc-daemon[890]: time=2026-09-20T09:33:47.635Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4199lb # [ 52.011559] niks3-server[1004]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4200lb # [ 52.025718] niks3-server[1004]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4201lb # [ 52.036175] niks3-server[1004]: 2026/09/20 09:33:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4202lb # [ 52.042936] niks3-server[1004]: 2026/09/20 09:33:48 INFO Signed narinfos id=10 count=14203worker1 # [ 52.084263] nix-grpc-daemon[890]: time=2026-09-20T09:33:47.701Z level=INFO msg="Uploading 1 narinfos"4204lb # [ 52.045416] niks3-server[1004]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4205lb # [ 52.060409] niks3-server[1004]: 2026/09/20 09:33:48 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4206lb # [ 52.067646] niks3-server[1004]: 2026/09/20 09:33:48 INFO Completed upload id=104207lb # [ 52.070927] niks3-server[1004]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4208worker1 # [ 52.108293] nix-grpc-daemon[890]: time=2026-09-20T09:33:47.726Z level=INFO msg="Upload complete. (347ms)"4209worker1 # [ 52.122440] nix-grpc-daemon[774]: ts=2026-09-20T09:33:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14210worker1 # [ 52.141921] nix-grpc-daemon[774]: ts=2026-09-20T09:33:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14211worker1 # [ 52.149529] nix-daemon[836]: reaped child process 916, status = succeeded4212client # [ 52.207778] systemd[1]: dedup2.service: Deactivated successfully.4213client # [ 52.212439] systemd[1]: dedup2.service: Consumed 186ms CPU time over 4.139s wall clock time, 33M memory peak, 7.3K incoming IP traffic, 6.3K outgoing IP traffic.4214client # [ 52.224941] systemd[1]: dedup1.service: Deactivated successfully.4215client # [ 52.227268] systemd[1]: dedup1.service: Consumed 157ms CPU time over 5.271s wall clock time, 37.2M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.4216client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4217client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4218client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4219worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4220worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4221worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4222worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4223worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4224worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.08 seconds)4225worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4226worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.09 seconds)4227(finished: subtest: two clients wanting the same drv build it once, in 6.34 seconds)4228subtest: build hook: nix-daemon with builders = grpc://lb4229client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4230client # [ 53.439371] systemd[1]: Started Nix Daemon.4231client # [ 53.628141] nix-daemon[928]: accepted connection from pid 925, user root (trusted)4232worker2 # [ 53.753267] nix-grpc-daemon[759]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364764233worker1 # [ 53.809675] nix-grpc-daemon[774]: ts=2026-09-20T09:33:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443444234worker1 # [ 53.828238] nix-grpc-daemon[774]: ts=2026-09-20T09:33:49Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04235worker1 # [ 53.834154] nix-grpc-daemon[774]: ts=2026-09-20T09:33:49Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14236worker2 # [ 53.830007] nix-grpc-daemon[759]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764237worker2 # [ 53.845506] nix-grpc-daemon[759]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764238worker2 # [ 53.860744] nix-grpc-daemon[759]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764239worker2 # [ 53.877965] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4240worker2 # [ 54.091107] nix-daemon[832]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4241worker2 # [ 54.111802] nix-daemon[832]: reaped child process 1034, status = succeeded4242worker2 # [ 54.284587] nix-daemon[832]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4243worker2 # [ 54.302016] nix-daemon[832]: reaped child process 1039, status = succeeded4244lb # [ 54.323894] niks3-server[1004]: 2026/09/20 09:33:50 INFO Received uploads request method=POST path=/api/pending_closures4245worker2 # [ 54.324952] nix-grpc-daemon[842]: time=2026-09-20T09:33:50.952Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4246worker2 # [ 54.326434] nix-grpc-daemon[842]: time=2026-09-20T09:33:50.952Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4247lb # [ 54.350686] niks3-server[1004]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4248lb # [ 54.365432] niks3-server[1004]: 2026/09/20 09:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4249worker2 # [ 54.362666] nix-grpc-daemon[842]: time=2026-09-20T09:33:50.990Z level=INFO msg="Uploading 1 narinfos"4250lb # [ 54.377171] niks3-server[1004]: 2026/09/20 09:33:50 INFO Signed narinfos id=11 count=14251lb # [ 54.384336] niks3-server[1004]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4252lb # [ 54.391800] niks3-server[1004]: 2026/09/20 09:33:51 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4253lb # [ 54.401824] niks3-server[1004]: 2026/09/20 09:33:51 INFO Completed upload id=114254worker2 # [ 54.391925] nix-grpc-daemon[842]: time=2026-09-20T09:33:51.019Z level=INFO msg="Upload complete. (496ms)"4255lb # [ 54.404500] niks3-server[1004]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4256worker2 # [ 54.400165] nix-grpc-daemon[759]: ts=2026-09-20T09:33:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=5124257worker2 # [ 54.408804] nix-grpc-daemon[759]: ts=2026-09-20T09:33:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764258worker2 # [ 54.420396] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4259worker2 # [ 54.425669] nix-daemon[832]: reaped child process 1029, status = succeeded4260worker2 # [ 54.438435] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4261worker2 # [ 55.289548] nix-daemon[832]: reaped child process 1043, status = succeeded4262worker2 # [ 55.462829] nix-daemon[832]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)4263worker2 # [ 55.483389] nix-daemon[832]: reaped child process 1061, status = succeeded4264lb # [ 55.505653] niks3-server[1004]: 2026/09/20 09:33:52 INFO Received uploads request method=POST path=/api/pending_closures4265worker2 # [ 55.508491] nix-grpc-daemon[842]: time=2026-09-20T09:33:52.133Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4266worker2 # [ 55.510031] nix-grpc-daemon[842]: time=2026-09-20T09:33:52.133Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4267lb # [ 55.552656] niks3-server[1004]: 2026/09/20 09:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4268worker2 # [ 55.555290] nix-grpc-daemon[842]: time=2026-09-20T09:33:52.183Z level=INFO msg="Uploading 1 narinfos"4269lb # [ 55.569197] niks3-server[1004]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4270lb # [ 55.582254] niks3-server[1004]: 2026/09/20 09:33:52 INFO Signed narinfos id=12 count=14271lb # [ 55.586431] niks3-server[1004]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4272lb # [ 55.595156] niks3-server[1004]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4273lb # [ 55.600384] niks3-server[1004]: 2026/09/20 09:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4274lb # [ 55.606300] niks3-server[1004]: 2026/09/20 09:33:52 INFO Completed upload id=124275worker2 # [ 55.589612] nix-grpc-daemon[842]: time=2026-09-20T09:33:52.217Z level=INFO msg="Upload complete. (337ms)"4276lb # [ 55.609268] niks3-server[1004]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4277worker2 # [ 55.601514] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14278worker1 # [ 55.661833] nix-grpc-daemon[774]: ts=2026-09-20T09:33:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443444279worker2 # [ 55.619522] nix-daemon[832]: reaped child process 1041, status = succeeded4280worker1 # [ 55.672432] nix-grpc-daemon[774]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4281worker2 # [ 55.665977] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364764282worker2 # [ 55.673082] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=1284283worker2 # [ 55.930295] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364764284worker2 # [ 55.937654] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764285worker2 # [ 55.944387] nix-grpc-daemon[842]: time=2026-09-20T09:33:52.571Z level=INFO msg="All 1 paths already cached"4286worker1 # [ 56.007718] nix-grpc-daemon[774]: ts=2026-09-20T09:33:51Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04287worker2 # [ 55.959297] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764288worker1 # [ 56.013681] nix-grpc-daemon[774]: ts=2026-09-20T09:33:51Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14289worker2 # [ 55.974088] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764290worker2 # [ 55.988836] nix-grpc-daemon[842]: time=2026-09-20T09:33:52.616Z level=INFO msg="All 1 paths already cached"4291worker2 # [ 55.998191] nix-grpc-daemon[759]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764292worker2 # [ 56.012379] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4293worker2 # [ 56.223430] nix-daemon[832]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)4294worker2 # [ 56.241480] nix-daemon[832]: reaped child process 1068, status = succeeded4295worker2 # [ 56.413822] nix-daemon[832]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4296worker2 # [ 56.431096] nix-daemon[832]: reaped child process 1073, status = succeeded4297lb # [ 56.452390] niks3-server[1004]: 2026/09/20 09:33:53 INFO Received uploads request method=POST path=/api/pending_closures4298worker2 # [ 56.452252] nix-grpc-daemon[842]: time=2026-09-20T09:33:53.080Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4299worker2 # [ 56.453759] nix-grpc-daemon[842]: time=2026-09-20T09:33:53.080Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4300lb # [ 56.484692] niks3-server[1004]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4301lb # [ 56.498547] niks3-server[1004]: 2026/09/20 09:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4302worker2 # [ 56.495736] nix-grpc-daemon[842]: time=2026-09-20T09:33:53.123Z level=INFO msg="Uploading 1 narinfos"4303lb # [ 56.514288] niks3-server[1004]: 2026/09/20 09:33:53 INFO Signed narinfos id=13 count=14304worker2 # [ 56.516855] nix-grpc-daemon[842]: time=2026-09-20T09:33:53.144Z level=INFO msg="Upload complete. (486ms)"4305lb # [ 56.521445] niks3-server[1004]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4306lb # [ 56.533491] niks3-server[1004]: 2026/09/20 09:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4307lb # [ 56.538302] niks3-server[1004]: 2026/09/20 09:33:53 INFO Completed upload id=134308worker2 # [ 56.528996] nix-grpc-daemon[759]: ts=2026-09-20T09:33:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=6244309lb # [ 56.540858] niks3-server[1004]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4310worker2 # [ 56.537582] nix-grpc-daemon[759]: ts=2026-09-20T09:33:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764311worker2 # [ 56.544321] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4312worker2 # [ 56.548791] nix-daemon[832]: reaped child process 1063, status = succeeded4313worker2 # [ 56.561182] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4314worker2 # [ 57.990028] nix-daemon[832]: reaped child process 1077, status = succeeded4315worker2 # [ 58.165979] nix-daemon[832]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)4316worker2 # [ 58.185869] nix-daemon[832]: reaped child process 1095, status = succeeded4317lb # [ 58.210507] niks3-server[1004]: 2026/09/20 09:33:54 INFO Received uploads request method=POST path=/api/pending_closures4318worker2 # [ 58.211231] nix-grpc-daemon[842]: time=2026-09-20T09:33:54.838Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4319worker2 # [ 58.212849] nix-grpc-daemon[842]: time=2026-09-20T09:33:54.838Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4320lb # [ 58.252438] niks3-server[1004]: 2026/09/20 09:33:54 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4321lb # [ 58.264521] niks3-server[1004]: 2026/09/20 09:33:54 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4322worker2 # [ 58.264218] nix-grpc-daemon[842]: time=2026-09-20T09:33:54.891Z level=INFO msg="Uploading 1 narinfos"4323lb # [ 58.276394] niks3-server[1004]: 2026/09/20 09:33:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4324lb # [ 58.285352] niks3-server[1004]: 2026/09/20 09:33:54 INFO Signed narinfos id=14 count=14325lb # [ 58.292964] niks3-server[1004]: 2026/09/20 09:33:54 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4326lb # [ 58.299489] niks3-server[1004]: 2026/09/20 09:33:54 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4327lb # [ 58.303661] niks3-server[1004]: 2026/09/20 09:33:54 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4328lb # [ 58.307763] niks3-server[1004]: 2026/09/20 09:33:54 INFO Completed upload id=144329worker2 # [ 58.293853] nix-grpc-daemon[842]: time=2026-09-20T09:33:54.921Z level=INFO msg="Upload complete. (322ms)"4330worker2 # [ 58.305424] nix-grpc-daemon[759]: ts=2026-09-20T09:33:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14331worker1 # [ 58.361272] nix-grpc-daemon[774]: ts=2026-09-20T09:33:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443444332worker2 # [ 58.323440] nix-daemon[832]: reaped child process 1075, status = succeeded4333worker1 # [ 58.374977] nix-grpc-daemon[774]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4334worker2 # [ 58.364651] nix-grpc-daemon[759]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364764335worker2 # [ 58.371455] nix-grpc-daemon[759]: ts=2026-09-20T09:33:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=1764336worker1 # [ 58.692351] nix-grpc-daemon[774]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:443444337worker1 # [ 58.700695] nix-grpc-daemon[774]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443444338worker1 # [ 58.708571] nix-grpc-daemon[890]: time=2026-09-20T09:33:54.324Z level=INFO msg="All 2 paths already cached"4339worker1 # [ 58.721276] nix-grpc-daemon[774]: ts=2026-09-20T09:33:54Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04340worker2 # [ 58.673250] nix-grpc-daemon[759]: ts=2026-09-20T09:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764341worker1 # [ 58.727487] nix-grpc-daemon[774]: ts=2026-09-20T09:33:54Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14342worker2 # [ 58.691558] nix-grpc-daemon[759]: ts=2026-09-20T09:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764343worker2 # [ 58.706084] nix-grpc-daemon[842]: time=2026-09-20T09:33:55.334Z level=INFO msg="All 2 paths already cached"4344worker2 # [ 58.715479] nix-grpc-daemon[759]: ts=2026-09-20T09:33:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764345worker2 # [ 58.730235] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4346worker2 # [ 58.938078] nix-daemon[832]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)4347worker2 # [ 58.958300] nix-daemon[832]: reaped child process 1102, status = succeeded4348worker2 # [ 59.138479] nix-daemon[832]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4349worker2 # [ 59.157583] nix-daemon[832]: reaped child process 1107, status = succeeded4350lb # [ 59.181035] niks3-server[1004]: 2026/09/20 09:33:55 INFO Received uploads request method=POST path=/api/pending_closures4351worker2 # [ 59.180806] nix-grpc-daemon[842]: time=2026-09-20T09:33:55.808Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4352worker2 # [ 59.182308] nix-grpc-daemon[842]: time=2026-09-20T09:33:55.808Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4353lb # [ 59.213536] niks3-server[1004]: 2026/09/20 09:33:55 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4354lb # [ 59.223810] niks3-server[1004]: 2026/09/20 09:33:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4355worker2 # [ 59.222399] nix-grpc-daemon[842]: time=2026-09-20T09:33:55.849Z level=INFO msg="Uploading 1 narinfos"4356lb # [ 59.234915] niks3-server[1004]: 2026/09/20 09:33:55 INFO Signed narinfos id=15 count=14357lb # [ 59.241372] niks3-server[1004]: 2026/09/20 09:33:55 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4358lb # [ 59.251147] niks3-server[1004]: 2026/09/20 09:33:55 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4359lb # [ 59.256512] niks3-server[1004]: 2026/09/20 09:33:55 INFO Completed upload id=154360worker2 # [ 59.246684] nix-grpc-daemon[842]: time=2026-09-20T09:33:55.874Z level=INFO msg="Upload complete. (497ms)"4361lb # [ 59.259326] niks3-server[1004]: 2026/09/20 09:33:55 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4362worker2 # [ 59.253597] nix-grpc-daemon[759]: ts=2026-09-20T09:33:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=7524363worker2 # [ 59.267770] nix-grpc-daemon[759]: ts=2026-09-20T09:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764364worker2 # [ 59.273874] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4365worker2 # [ 59.278629] nix-daemon[832]: reaped child process 1097, status = succeeded4366worker2 # [ 59.293207] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4367lb # [ 59.397608] envoy[902]: conn peer=[2001:db8:1::1]:56644 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=37574368lb # [ 59.402636] envoy[902]: conn peer=[2001:db8:1::1]:56636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1372 ms=49594369lb # [ 59.407545] envoy[902]: conn peer=[2001:db8:1::1]:56000 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=304370lb # [ 59.412344] envoy[902]: conn peer=[2001:db8:1::1]:55988 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3388 tx=1777 ms=19404371lb # [ 59.417126] envoy[902]: conn peer=[2001:db8:1::1]:56004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3874 tx=1990 ms=24584372lb # [ 59.421844] envoy[902]: conn peer=[2001:db8:1::1]:56018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=284373worker2 # [ 60.122247] nix-daemon[832]: reaped child process 1111, status = succeeded4374worker2 # [ 60.285990] nix-daemon[832]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)4375worker2 # [ 60.303999] nix-daemon[832]: reaped child process 1129, status = succeeded4376lb # [ 60.326824] niks3-server[1004]: 2026/09/20 09:33:56 INFO Received uploads request method=POST path=/api/pending_closures4377worker2 # [ 60.328407] nix-grpc-daemon[842]: time=2026-09-20T09:33:56.954Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4378worker2 # [ 60.329890] nix-grpc-daemon[842]: time=2026-09-20T09:33:56.954Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4379lb # [ 60.363074] niks3-server[1004]: 2026/09/20 09:33:56 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4380lb # [ 60.380493] niks3-server[1004]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4381lb # [ 60.389682] niks3-server[1004]: 2026/09/20 09:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4382worker2 # [ 60.384452] nix-grpc-daemon[842]: time=2026-09-20T09:33:57.012Z level=INFO msg="Uploading 1 narinfos"4383lb # [ 60.399604] niks3-server[1004]: 2026/09/20 09:33:57 INFO Signed narinfos id=16 count=14384lb # [ 60.404757] niks3-server[1004]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4385lb # [ 60.414871] niks3-server[1004]: 2026/09/20 09:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4386lb # [ 60.418916] niks3-server[1004]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4387lb # [ 60.423447] niks3-server[1004]: 2026/09/20 09:33:57 INFO Completed upload id=164388worker2 # [ 60.413328] nix-grpc-daemon[842]: time=2026-09-20T09:33:57.041Z level=INFO msg="Upload complete. (310ms)"4389worker2 # [ 60.426423] nix-grpc-daemon[759]: ts=2026-09-20T09:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14390worker2 # [ 60.441312] nix-grpc-daemon[759]: ts=2026-09-20T09:33:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364764391worker2 # [ 60.445218] nix-daemon[832]: reaped child process 1109, status = succeeded4392worker1 # [ 60.505997] nix-grpc-daemon[774]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443444393worker1 # [ 60.514763] nix-grpc-daemon[774]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4394worker1 # [ 60.533473] nix-grpc-daemon[774]: ts=2026-09-20T09:33:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=0 paths=1 nar_bytes_out=2324395client # [ 60.691348] nix-daemon[928]: reaped child process 930, status = succeeded4396client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.49 seconds)4397client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4398client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4399client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4400client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.06 seconds)4401client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4402client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4403(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.63 seconds)4404subtest: an input only one worker has still reaches the builder4405client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4406client # this derivation will be built:4407client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4408client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4409client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.43 seconds)4410client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4411client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4412worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4413worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4414worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4415worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4416client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24417client # [ 61.996926] nix-daemon[928]: accepted connection from pid 1007, user root (trusted)4418client # this derivation will be built:4419client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4420client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4421worker1 # [ 62.173370] nix-grpc-daemon[774]: ts=2026-09-20T09:33:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:443444422worker2 # [ 62.126791] nix-grpc-daemon[759]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764423client # copying 1 paths...4424client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4425worker2 # [ 62.143077] nix-grpc-daemon[759]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764426worker2 # [ 62.158176] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4427worker2 # [ 62.381183] nix-daemon[832]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)4428worker2 # [ 62.400312] nix-daemon[832]: reaped child process 1139, status = succeeded4429lb # [ 62.422435] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received uploads request method=POST path=/api/pending_closures4430worker2 # [ 62.422166] nix-grpc-daemon[842]: time=2026-09-20T09:33:59.049Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4431worker2 # [ 62.423689] nix-grpc-daemon[842]: time=2026-09-20T09:33:59.049Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4432lb # [ 62.450242] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4433lb # [ 62.464902] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4434worker2 # [ 62.461214] nix-grpc-daemon[842]: time=2026-09-20T09:33:59.089Z level=INFO msg="Uploading 1 narinfos"4435lb # [ 62.477246] niks3-server[1004]: 2026/09/20 09:33:59 INFO Signed narinfos id=17 count=14436lb # [ 62.482605] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4437lb # [ 62.492098] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4438worker2 # [ 62.481998] nix-grpc-daemon[842]: time=2026-09-20T09:33:59.109Z level=INFO msg="Upload complete. (307ms)"4439lb # [ 62.496835] niks3-server[1004]: 2026/09/20 09:33:59 INFO Completed upload id=174440worker2 # [ 62.488484] nix-grpc-daemon[759]: ts=2026-09-20T09:33:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=1204441lb # [ 62.499674] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4442worker2 # [ 62.502672] nix-daemon[832]: reaped child process 1134, status = succeeded4443worker1 # [ 62.557319] nix-grpc-daemon[774]: ts=2026-09-20T09:33:58Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04444worker1 # [ 62.565071] nix-grpc-daemon[774]: ts=2026-09-20T09:33:58Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14445worker1 # [ 62.606705] nix-grpc-daemon[774]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443444446worker1 # [ 62.622953] nix-grpc-daemon[774]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443444447client # copying 2 paths to 'lb:50051'...4448client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4449worker1 # [ 62.643166] nix-grpc-daemon[774]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443444450client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4451worker1 # [ 62.658852] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4452worker1 # [ 62.862398] nix-daemon[836]: accepted connection from pid 980, user nix-grpc-daemon (trusted)4453worker1 # [ 62.879898] nix-daemon[836]: reaped child process 983, status = succeeded4454worker1 # [ 63.052911] nix-daemon[836]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4455worker1 # [ 63.071021] nix-daemon[836]: reaped child process 988, status = succeeded4456lb # [ 63.040215] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received uploads request method=POST path=/api/pending_closures4457lb # [ 63.050170] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received uploads request method=POST path=/api/pending_closures4458worker1 # [ 63.101718] nix-grpc-daemon[890]: time=2026-09-20T09:33:58.718Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4459worker1 # [ 63.103366] nix-grpc-daemon[890]: time=2026-09-20T09:33:58.718Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4460worker1 # [ 63.105266] nix-grpc-daemon[890]: time=2026-09-20T09:33:58.718Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4461lb # [ 63.088655] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4462lb # [ 63.100441] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4463lb # [ 63.115320] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4464worker1 # [ 63.162591] nix-grpc-daemon[890]: time=2026-09-20T09:33:58.779Z level=INFO msg="Uploading 3 narinfos"4465lb # [ 63.123438] niks3-server[1004]: 2026/09/20 09:33:59 INFO Signed narinfos id=18 count=14466lb # [ 63.128327] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4467lb # [ 63.131700] niks3-server[1004]: 2026/09/20 09:33:59 INFO Signed narinfos id=19 count=24468lb # [ 63.140541] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4469lb # [ 63.143503] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4470lb # [ 63.156098] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4471lb # [ 63.166164] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4472lb # [ 63.168830] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4473lb # [ 63.171572] niks3-server[1004]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4474lb # [ 63.175341] niks3-server[1004]: 2026/09/20 09:33:59 INFO Completed upload id=184475lb # [ 63.177281] niks3-server[1004]: 2026/09/20 09:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4476lb # [ 63.180187] niks3-server[1004]: 2026/09/20 09:33:59 INFO Completed upload id=194477worker1 # [ 63.221638] nix-grpc-daemon[890]: time=2026-09-20T09:33:58.839Z level=INFO msg="Upload complete. (542ms)"4478worker1 # [ 63.233504] nix-grpc-daemon[774]: ts=2026-09-20T09:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=0 paths=2 nar_bytes_in=9844479worker1 # [ 63.252649] nix-grpc-daemon[774]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443444480worker1 # [ 63.261315] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4481worker1 # [ 63.267402] nix-daemon[836]: reaped child process 978, status = succeeded4482client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4483worker1 # [ 63.281886] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4484client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4485worker1 # [ 64.131922] nix-daemon[836]: reaped child process 992, status = succeeded4486worker1 # [ 64.308307] nix-daemon[836]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4487worker1 # [ 64.327582] nix-daemon[836]: reaped child process 1010, status = succeeded4488lb # [ 64.299618] niks3-server[1004]: 2026/09/20 09:34:00 INFO Received uploads request method=POST path=/api/pending_closures4489worker1 # [ 64.351590] nix-grpc-daemon[890]: time=2026-09-20T09:33:59.968Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4490lb # [ 64.333445] niks3-server[1004]: 2026/09/20 09:34:00 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4491worker1 # [ 64.385919] nix-grpc-daemon[890]: time=2026-09-20T09:34:00.003Z level=INFO msg="Uploading 1 narinfos"4492lb # [ 64.346833] niks3-server[1004]: 2026/09/20 09:34:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4493lb # [ 64.355597] niks3-server[1004]: 2026/09/20 09:34:00 INFO Signed narinfos id=20 count=14494lb # [ 64.363713] niks3-server[1004]: 2026/09/20 09:34:00 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4495lb # [ 64.371164] niks3-server[1004]: 2026/09/20 09:34:00 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4496worker1 # [ 64.419242] nix-grpc-daemon[890]: time=2026-09-20T09:34:00.037Z level=INFO msg="Upload complete. (322ms)"4497lb # [ 64.379653] niks3-server[1004]: 2026/09/20 09:34:01 INFO Completed upload id=204498lb # [ 64.382628] niks3-server[1004]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4499client # copying 1 paths...4500worker1 # [ 64.433584] nix-grpc-daemon[774]: ts=2026-09-20T09:34:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14501client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4502worker1 # [ 64.456120] nix-grpc-daemon[774]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443444503worker1 # [ 64.460480] nix-daemon[836]: reaped child process 990, status = succeeded4504worker1 # [ 64.478635] nix-grpc-daemon[774]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443444505worker1 # [ 64.484597] nix-grpc-daemon[774]: ts=2026-09-20T09:34:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=0 paths=1 nar_bytes_out=1204506client # [ 64.639463] nix-daemon[928]: reaped child process 1010, status = succeeded4507client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.88 seconds)4508client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24509client # [ 64.865059] nix-daemon[928]: accepted connection from pid 1028, user root (trusted)4510client # this derivation will be built:4511client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4512worker2 # [ 65.001128] nix-grpc-daemon[759]: ts=2026-09-20T09:34:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364764513client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4514worker2 # [ 65.007821] nix-grpc-daemon[759]: ts=2026-09-20T09:34:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764515worker2 # [ 65.021770] nix-grpc-daemon[842]: time=2026-09-20T09:34:01.647Z level=INFO msg="All 1 paths already cached"4516worker1 # [ 65.086899] nix-grpc-daemon[774]: ts=2026-09-20T09:34:00Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04517worker1 # [ 65.093942] nix-grpc-daemon[774]: ts=2026-09-20T09:34:00Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14518worker2 # [ 65.047063] nix-grpc-daemon[759]: ts=2026-09-20T09:34:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764519worker2 # [ 65.067101] nix-grpc-daemon[759]: ts=2026-09-20T09:34:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764520client # copying 1 paths to 'lb:50051'...4521client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4522worker2 # [ 65.096977] nix-grpc-daemon[759]: ts=2026-09-20T09:34:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764523worker2 # [ 65.113955] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4524worker2 # [ 65.140883] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)4525worker2 # [ 65.246729] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 2/5)4526worker2 # [ 65.257663] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 3/5)4527worker2 # [ 65.407169] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 621 ms (attempt 4/5)4528worker2 # [ 66.039396] nix-grpc-daemon[759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4529worker2 # [ 66.047090] nix-grpc-daemon[759]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4530worker2 # [ 66.295013] nix-daemon[832]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)4531worker2 # [ 66.315996] nix-daemon[832]: reaped child process 1150, status = succeeded4532worker2 # [ 66.503259] nix-daemon[832]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)4533worker2 # [ 66.521531] nix-daemon[832]: reaped child process 1155, status = succeeded4534lb # [ 66.547832] niks3-server[1004]: 2026/09/20 09:34:03 INFO Received uploads request method=POST path=/api/pending_closures4535worker2 # [ 66.548518] nix-grpc-daemon[842]: time=2026-09-20T09:34:03.175Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4536worker2 # [ 66.550001] nix-grpc-daemon[842]: time=2026-09-20T09:34:03.175Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4537lb # [ 66.596727] niks3-server[1004]: 2026/09/20 09:34:03 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4538lb # [ 66.616294] niks3-server[1004]: 2026/09/20 09:34:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4539worker2 # [ 66.612977] nix-grpc-daemon[842]: time=2026-09-20T09:34:03.240Z level=INFO msg="Uploading 1 narinfos"4540lb # [ 66.626612] niks3-server[1004]: 2026/09/20 09:34:03 INFO Signed narinfos id=21 count=14541lb # [ 66.631710] niks3-server[1004]: 2026/09/20 09:34:03 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4542lb # [ 66.640124] niks3-server[1004]: 2026/09/20 09:34:03 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4543lb # [ 66.647535] niks3-server[1004]: 2026/09/20 09:34:03 INFO Completed upload id=214544worker2 # [ 66.637119] nix-grpc-daemon[842]: time=2026-09-20T09:34:03.265Z level=INFO msg="Upload complete. (556ms)"4545lb # [ 66.650006] niks3-server[1004]: 2026/09/20 09:34:03 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4546worker2 # [ 66.648764] nix-grpc-daemon[759]: ts=2026-09-20T09:34:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 paths=1 nar_bytes_in=5684547worker2 # [ 66.663598] nix-grpc-daemon[759]: ts=2026-09-20T09:34:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764548worker2 # [ 66.674822] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4549worker2 # [ 66.682387] nix-daemon[832]: reaped child process 1141, status = succeeded4550client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4551worker2 # [ 66.695863] nix-daemon[832]: accepted connection from pid 759, user nix-grpc-daemon (trusted)4552client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4553worker2 # [ 68.155641] nix-daemon[832]: reaped child process 1159, status = succeeded4554worker2 # [ 68.328723] nix-daemon[832]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)4555worker2 # [ 68.349440] nix-daemon[832]: reaped child process 1177, status = succeeded4556lb # [ 68.371125] niks3-server[1004]: 2026/09/20 09:34:04 INFO Received uploads request method=POST path=/api/pending_closures4557worker2 # [ 68.372445] nix-grpc-daemon[842]: time=2026-09-20T09:34:04.998Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4558lb # [ 68.399012] niks3-server[1004]: 2026/09/20 09:34:05 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4559worker2 # [ 68.409848] nix-grpc-daemon[842]: time=2026-09-20T09:34:05.037Z level=INFO msg="Uploading 1 narinfos"4560lb # [ 68.412856] niks3-server[1004]: 2026/09/20 09:34:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4561lb # [ 68.424670] niks3-server[1004]: 2026/09/20 09:34:05 INFO Signed narinfos id=22 count=14562lb # [ 68.431692] niks3-server[1004]: 2026/09/20 09:34:05 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4563lb # [ 68.441345] niks3-server[1004]: 2026/09/20 09:34:05 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4564lb # [ 68.449674] niks3-server[1004]: 2026/09/20 09:34:05 INFO Completed upload id=224565worker2 # [ 68.440047] nix-grpc-daemon[842]: time=2026-09-20T09:34:05.068Z level=INFO msg="Upload complete. (331ms)"4566lb # [ 68.453114] niks3-server[1004]: 2026/09/20 09:34:05 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4567worker2 # [ 68.453114] nix-grpc-daemon[759]: ts=2026-09-20T09:34:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14568worker2 # [ 68.467930] nix-daemon[832]: reaped child process 1157, status = succeeded4569client # copying 1 paths...4570worker1 # [ 68.553676] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443444571worker1 # [ 68.569086] nix-grpc-daemon[774]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4572client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4573worker1 # [ 68.616151] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443444574worker1 # [ 68.622582] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=0 paths=1 nar_bytes_out=1204575client # [ 68.783163] nix-daemon[928]: reaped child process 1031, status = succeeded4576client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 4.14 seconds)4577client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24578client # [ 69.001558] nix-daemon[928]: accepted connection from pid 1049, user root (trusted)4579client # this derivation will be built:4580client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4581worker1 # [ 69.180914] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:443444582client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4583worker1 # [ 69.190553] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443444584worker1 # [ 69.198684] nix-grpc-daemon[890]: time=2026-09-20T09:34:04.814Z level=INFO msg="All 1 paths already cached"4585worker1 # [ 69.211804] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04586worker1 # [ 69.219682] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14587worker1 # [ 69.229547] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443444588worker1 # [ 69.241418] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443444589worker1 # [ 69.256693] nix-grpc-daemon[890]: time=2026-09-20T09:34:04.874Z level=INFO msg="All 1 paths already cached"4590client # copying 1 paths to 'lb:50051'...4591client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4592worker1 # [ 69.268058] nix-grpc-daemon[774]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443444593worker1 # [ 69.285682] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4594lb # [ 69.400571] envoy[902]: conn peer=[2001:db8:1::1]:56020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4352 tx=2086 ms=18994595lb # [ 69.400923] envoy[902]: conn peer=[2001:db8:1::1]:56022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=934596lb # [ 69.401069] envoy[902]: conn peer=[2001:db8:1::1]:53932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4219 tx=1770 ms=23724597lb # [ 69.401174] envoy[902]: conn peer=[2001:db8:1::1]:53936 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=724598lb # [ 69.401276] envoy[902]: conn peer=[2001:db8:1::1]:53950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=694599lb # [ 69.401402] envoy[902]: conn peer=[2001:db8:1::1]:53948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3723 tx=1916 ms=36324600worker1 # [ 69.498609] nix-daemon[836]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4601worker1 # [ 69.517860] nix-daemon[836]: reaped child process 1022, status = succeeded4602worker1 # [ 69.691655] nix-daemon[836]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4603worker1 # [ 69.708914] nix-daemon[836]: reaped child process 1027, status = succeeded4604lb # [ 69.683953] niks3-server[1004]: 2026/09/20 09:34:06 INFO Received uploads request method=POST path=/api/pending_closures4605worker1 # [ 69.733667] nix-grpc-daemon[890]: time=2026-09-20T09:34:05.351Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4606worker1 # [ 69.735167] nix-grpc-daemon[890]: time=2026-09-20T09:34:05.351Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4607lb # [ 69.711699] niks3-server[1004]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4608worker1 # [ 69.768127] nix-grpc-daemon[890]: time=2026-09-20T09:34:05.386Z level=INFO msg="Uploading 1 narinfos"4609lb # [ 69.726902] niks3-server[1004]: 2026/09/20 09:34:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4610lb # [ 69.739549] niks3-server[1004]: 2026/09/20 09:34:06 INFO Signed narinfos id=23 count=14611lb # [ 69.746484] niks3-server[1004]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4612lb # [ 69.754783] niks3-server[1004]: 2026/09/20 09:34:06 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4613lb # [ 69.763995] niks3-server[1004]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4614lb # [ 69.768906] niks3-server[1004]: 2026/09/20 09:34:06 INFO Completed upload id=234615worker1 # [ 69.810404] nix-grpc-daemon[890]: time=2026-09-20T09:34:05.428Z level=INFO msg="Upload complete. (505ms)"4616worker1 # [ 69.820784] nix-grpc-daemon[774]: ts=2026-09-20T09:34:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=0 paths=1 nar_bytes_in=5684617worker1 # [ 69.834847] nix-grpc-daemon[774]: ts=2026-09-20T09:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443444618worker1 # [ 69.843242] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4619worker1 # [ 69.847059] nix-daemon[836]: reaped child process 1017, status = succeeded4620client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4621worker1 # [ 69.858489] nix-daemon[836]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4622client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4623worker1 # [ 70.905063] nix-daemon[836]: reaped child process 1031, status = succeeded4624worker1 # [ 71.088945] nix-daemon[836]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4625worker1 # [ 71.106106] nix-daemon[836]: reaped child process 1049, status = succeeded4626lb # [ 71.077660] niks3-server[1004]: 2026/09/20 09:34:07 INFO Received uploads request method=POST path=/api/pending_closures4627worker1 # [ 71.128413] nix-grpc-daemon[890]: time=2026-09-20T09:34:06.746Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4628lb # [ 71.111102] niks3-server[1004]: 2026/09/20 09:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4629worker1 # [ 71.163214] nix-grpc-daemon[890]: time=2026-09-20T09:34:06.781Z level=INFO msg="Uploading 1 narinfos"4630lb # [ 71.126641] niks3-server[1004]: 2026/09/20 09:34:07 INFO Signed narinfos id=24 count=14631lb # [ 71.134389] niks3-server[1004]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4632lb # [ 71.142135] niks3-server[1004]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4633lb # [ 71.149312] niks3-server[1004]: 2026/09/20 09:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4634worker1 # [ 71.195381] nix-grpc-daemon[890]: time=2026-09-20T09:34:06.813Z level=INFO msg="Upload complete. (323ms)"4635lb # [ 71.155536] niks3-server[1004]: 2026/09/20 09:34:07 INFO Completed upload id=244636lb # [ 71.159108] niks3-server[1004]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4637worker1 # [ 71.204898] nix-grpc-daemon[774]: ts=2026-09-20T09:34:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14638worker1 # [ 71.218538] nix-daemon[836]: reaped child process 1029, status = succeeded4639client # copying 1 paths...4640worker1 # [ 71.250712] nix-grpc-daemon[774]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443444641client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4642worker1 # [ 71.278822] nix-grpc-daemon[774]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443444643worker1 # [ 71.284089] nix-grpc-daemon[774]: ts=2026-09-20T09:34:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44344 duration_s=0 paths=1 nar_bytes_out=1204644client # [ 71.431818] nix-daemon[928]: reaped child process 1052, status = succeeded4645client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.65 seconds)4646(finished: subtest: an input only one worker has still reaches the builder, in 10.60 seconds)4647subtest: upload whose reference the worker lacks is completed from the cache4648client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4649client # this derivation will be built:4650client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4651client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4652client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.45 seconds)4653client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4654client # this derivation will be built:4655client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4656client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4657client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)4658client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4659client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4660worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4661worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4662worker2 # [ 72.775518] systemd[1]: Stopping Nix worker-protocol over gRPC...4663worker2 # [ 73.731947] nix-grpc-daemon[759]: ts=2026-09-20T09:34:10Z level=info event=signal_exit4664worker1 # [ 75.024329] nix-grpc-daemon[774]: ts=2026-09-20T09:34:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443324665worker2 # [ 74.977188] nix-grpc-daemon[759]: ts=2026-09-20T09:34:11Z level=info event=scheduler_restarting4666worker1 # [ 75.024920] nix-grpc-daemon[774]: ts=2026-09-20T09:34:10Z level=info event=worker_gone addr=192.168.1.4:500514667worker2 # [ 75.006040] nix-grpc-daemon[759]: ts=2026-09-20T09:34:11Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4668worker2 # [ 75.036157] nix-daemon[832]: reaped child process 834, status = succeeded4669worker2 # [ 75.208933] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4670worker2 # [ 75.211950] systemd[1]: Stopped Nix worker-protocol over gRPC.4671worker2 # [ 75.223266] systemd[1]: nix-grpc-daemon.service: Consumed 3.598s CPU time over 55.822s wall clock time, 62.2M memory peak, 84.3K incoming IP traffic, 154.7K outgoing IP traffic.4672worker2 # [ 75.231260] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4673worker2 # [ 75.236750] systemd[1]: Closed Nix worker-protocol over gRPC.4674worker2 # [ 75.239278] systemd[1]: nix-grpc-daemon.socket: Consumed 566us CPU time over 1min 3.766s wall clock time, 256K memory peak, 71.6K incoming IP traffic, 59.9K outgoing IP traffic.4675lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4676lb: must succeed: curl -sf localhost:9901/clusters4677lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4678lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4679client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24680worker1 # [ 75.629930] nix-grpc-daemon[774]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443444681worker1 # [ 75.638405] nix-grpc-daemon[774]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443444682client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4683client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.29 seconds)4684worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4685worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4686worker2 # [ 75.725779] systemd[1]: Listening on Nix worker-protocol over gRPC.4687worker1 # [ 75.864659] systemd[1]: Stopping Nix worker-protocol over gRPC...4688worker2 # [ 75.828721] systemd[1]: Starting Nix worker-protocol over gRPC...4689worker2 # [ 76.041690] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4690worker2 # [ 76.046090] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14691worker2 # [ 76.054600] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4692worker2 # [ 76.074890] systemd[1]: Started Nix worker-protocol over gRPC.4693worker2 # [ 76.099151] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:12Z level=info event=scheduler_connected addr=lb:500514694worker1 # [ 76.153689] nix-grpc-daemon[774]: ts=2026-09-20T09:34:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04695worker1 # [ 76.796261] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=signal_exit4696worker1 # [ 76.846596] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=scheduler_yield4697lb # [ 76.811660] niks3-server[1004]: 2026/09/20 09:34:13 INFO lead: released remote=192.168.1.3:431704698worker2 # [ 76.802378] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:13Z level=info event=scheduler_restarting addr=lb:500514699worker1 # [ 76.866010] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443324700worker1 # [ 76.872708] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=worker_gone addr=192.168.1.4:500514701worker1 # [ 76.876861] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=scheduler_restarting addr=lb:500514702worker1 # [ 76.880806] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:443324703worker1 # [ 76.885136] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=worker_gone addr=192.168.1.3:500514704worker2 # [ 77.003485] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:13Z level=info event=scheduler_connected addr=lb:500514705worker2 # [ 77.009650] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4706worker1 # [ 77.073203] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=scheduler_connected addr=lb:500514707worker1 # [ 77.078417] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4708worker2 # [ 77.211238] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:13Z level=info event=scheduler_connected addr=lb:500514709worker2 # [ 77.216858] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4710worker1 # [ 77.281127] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=scheduler_connected addr=lb:500514711worker1 # [ 77.285283] nix-grpc-daemon[774]: ts=2026-09-20T09:34:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4712worker2 # [ 77.417745] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_connected addr=lb:500514713worker2 # [ 77.423375] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4714worker1 # [ 77.481494] nix-grpc-daemon[774]: ts=2026-09-20T09:34:13Z level=info event=scheduler_connected addr=lb:500514715worker1 # [ 77.486686] nix-grpc-daemon[774]: ts=2026-09-20T09:34:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4716worker2 # [ 77.624066] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_connected addr=lb:500514717worker2 # [ 77.629431] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4718worker1 # [ 77.687632] nix-grpc-daemon[774]: ts=2026-09-20T09:34:13Z level=info event=scheduler_connected addr=lb:500514719worker1 # [ 77.692861] nix-grpc-daemon[774]: ts=2026-09-20T09:34:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4720worker1 # [ 77.852293] nix-grpc-daemon[774]: ts=2026-09-20T09:34:13Z level=info event=scheduler_restarting4721worker2 # [ 77.830239] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_connected addr=lb:500514722worker1 # [ 77.887828] nix-daemon[836]: reaped child process 838, status = succeeded4723worker2 # [ 77.835555] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4724worker1 # [ 77.918161] nix-grpc-daemon[774]: ts=2026-09-20T09:34:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4725worker1 # [ 77.928432] nix-grpc-daemon[774]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4726worker1 # [ 77.932656] nix-grpc-daemon[774]: E0000 00:00:1789896853.546256 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4727worker1 # [ 77.939217] nix-grpc-daemon[774]: E0000 00:00:1789896853.557186 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4728worker2 # [ 78.036423] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_connected addr=lb:500514729worker2 # [ 78.041574] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4730worker2 # [ 78.244353] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_connected addr=lb:500514731worker2 # [ 78.248456] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4732worker2 # [ 78.445370] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_connected addr=lb:500514733worker2 # [ 78.450600] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4734worker2 # [ 78.651481] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_connected addr=lb:500514735worker2 # [ 78.657589] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4736worker2 # [ 78.858470] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_connected addr=lb:500514737worker2 # [ 78.863909] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4738worker2 # [ 79.064874] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_connected addr=lb:500514739worker2 # [ 79.070352] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4740worker1 # [ 79.163916] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4741worker1 # [ 79.170721] systemd[1]: Stopped Nix worker-protocol over gRPC.4742worker1 # [ 79.177166] systemd[1]: nix-grpc-daemon.service: Consumed 1.934s CPU time over 59.737s wall clock time, 55.3M memory peak, 53.3K incoming IP traffic, 98.4K outgoing IP traffic.4743worker1 # [ 79.191582] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4744worker1 # [ 79.199874] systemd[1]: Closed Nix worker-protocol over gRPC.4745worker1 # [ 79.202541] systemd[1]: nix-grpc-daemon.socket: Consumed 570us CPU time over 1min 7.792s wall clock time, 256K memory peak, 76K incoming IP traffic, 62.6K outgoing IP traffic.4746lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4747lb: must succeed: curl -sf localhost:9901/clusters4748lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4749lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4750client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24751worker2 # [ 79.273093] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_connected addr=lb:500514752worker2 # [ 79.277347] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4753lb # [ 79.397914] envoy[902]: conn peer=[2001:db8:1::1]:39344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=624754lb # [ 79.402901] envoy[902]: conn peer=[2001:db8:1::1]:53958 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=1801 ms=21584755lb # [ 79.407842] envoy[902]: conn peer=[2001:db8:1::4]:46968 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=7724 tx=2315 ms=552274756lb # [ 79.412997] envoy[902]: conn peer=[2001:db8:1::1]:39346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=284757lb # [ 79.417643] envoy[902]: conn peer=[2001:db8:1::3]:55648 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=7596 tx=2108 ms=580654758worker2 # [ 79.473793] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500514759worker2 # [ 79.479400] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4760worker2 # [ 79.492749] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:390924761worker2 # [ 79.509207] nix-daemon[832]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)4762worker2 # [ 79.532212] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)4763worker2 # [ 79.627640] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 188 ms (attempt 2/5)4764worker2 # [ 79.680243] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500514765worker2 # [ 79.685363] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4766worker2 # [ 79.823508] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 238 ms (attempt 3/5)4767worker2 # [ 79.886172] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500514768worker2 # [ 79.891288] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4769worker2 # [ 80.069163] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 391 ms (attempt 4/5)4770worker2 # [ 80.092153] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500514771worker2 # [ 80.097159] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4772worker2 # [ 80.299113] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500514773worker2 # [ 80.304357] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4774worker2 # [ 80.468339] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4775client # copying 2 paths...4776client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4777worker2 # [ 80.495838] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:390924778client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4779worker2 # [ 80.506464] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500514780worker2 # [ 80.511656] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4781worker2 # [ 80.526387] nix-daemon[832]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)4782worker2 # [ 80.556648] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=niks3_push_started pid=12544783worker2 # [ 80.707243] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500514784worker2 # [ 80.728330] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4785worker2 # [ 80.913108] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500514786worker2 # [ 80.930172] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4787worker2 # [ 80.948284] nix-daemon[832]: accepted connection from pid 1267, user nix-grpc-daemon (trusted)4788worker2 # [ 80.966532] nix-daemon[832]: reaped child process 1270, status = succeeded4789lb # [ 80.989848] niks3-server[1004]: 2026/09/20 09:34:17 INFO Received uploads request method=POST path=/api/pending_closures4790lb # [ 81.000748] niks3-server[1004]: 2026/09/20 09:34:17 INFO Received uploads request method=POST path=/api/pending_closures4791worker2 # [ 81.002617] nix-grpc-daemon[1254]: time=2026-09-20T09:34:17.627Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4792worker2 # [ 81.005322] nix-grpc-daemon[1254]: time=2026-09-20T09:34:17.628Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4793worker2 # [ 81.007162] nix-grpc-daemon[1254]: time=2026-09-20T09:34:17.628Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4794lb # [ 81.050380] niks3-server[1004]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4795lb # [ 81.063000] niks3-server[1004]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4796lb # [ 81.076211] niks3-server[1004]: 2026/09/20 09:34:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4797worker2 # [ 81.075828] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_take_over4798lb # [ 81.090958] niks3-server[1004]: 2026/09/20 09:34:17 INFO Signed narinfos id=25 count=14799lb # [ 81.097231] niks3-server[1004]: 2026/09/20 09:34:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4800lb # [ 81.105708] niks3-server[1004]: 2026/09/20 09:34:17 INFO lead: acquired remote=192.168.1.4:422664801lb # [ 81.110131] niks3-server[1004]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4802worker2 # [ 81.099887] nix-grpc-daemon[1254]: time=2026-09-20T09:34:17.727Z level=INFO msg="Uploading 3 narinfos"4803lb # [ 81.113364] niks3-server[1004]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4804lb # [ 81.117523] niks3-server[1004]: 2026/09/20 09:34:17 INFO Signed narinfos id=26 count=24805worker2 # [ 81.115924] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500514806worker2 # [ 81.125918] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04807lb # [ 81.144923] niks3-server[1004]: 2026/09/20 09:34:17 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4808lb # [ 81.153598] niks3-server[1004]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4809lb # [ 81.156291] niks3-server[1004]: 2026/09/20 09:34:17 INFO Completed upload id=254810lb # [ 81.160443] niks3-server[1004]: 2026/09/20 09:34:17 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4811lb # [ 81.163257] niks3-server[1004]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4812lb # [ 81.167029] niks3-server[1004]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4813lb # [ 81.170035] niks3-server[1004]: 2026/09/20 09:34:17 INFO Completed upload id=264814worker2 # [ 81.160227] nix-grpc-daemon[1254]: time=2026-09-20T09:34:17.788Z level=INFO msg="Upload complete. (389ms)"4815worker2 # [ 81.172248] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39092 duration_s=0 paths=2 nar_bytes_in=3124816worker2 # [ 81.187150] nix-daemon[832]: reaped child process 1252, status = succeeded4817client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.97 seconds)4818worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4819worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4820worker1 # [ 81.384993] systemd[1]: Listening on Nix worker-protocol over gRPC.4821worker1 # [ 81.392744] systemd[1]: Starting Nix worker-protocol over gRPC...4822worker1 # [ 81.593194] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4823worker1 # [ 81.595866] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14824worker1 # [ 81.604064] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4825worker1 # [ 81.611993] systemd[1]: Started Nix worker-protocol over gRPC.4826lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4827lb: must succeed: curl -sf localhost:9901/clusters4828worker1 # [ 81.649573] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500514829worker2 # [ 81.603049] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04830lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)4831lb: must succeed: curl -sf localhost:9901/clusters4832lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4833lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.20 seconds)4834(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.41 seconds)4835subtest: scheduler down: the other node takes the lock and keeps it4836lb: must succeed: curl -sf localhost:9901/clusters4837lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4838lb: must succeed: curl -sf localhost:9901/clusters4839lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4840lb: must succeed: curl -sf localhost:9901/clusters4841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4842lb: must succeed: curl -sf localhost:9901/clusters4843lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4844worker2 # [ 83.054717] systemd[1]: Stopping Nix worker-protocol over gRPC...4845worker2 # [ 83.066447] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:19Z level=info event=signal_exit4846worker2 # [ 83.272667] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:19Z level=info event=scheduler_yield4847worker1 # [ 83.328694] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:18Z level=info event=scheduler_restarting addr=lb:500514848worker2 # [ 83.278589] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:427104849worker2 # [ 83.283312] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:19Z level=info event=worker_gone addr=192.168.1.3:500514850lb # [ 83.294833] niks3-server[1004]: 2026/09/20 09:34:19 INFO lead: released remote=192.168.1.4:422664851worker2 # [ 83.289498] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:19Z level=info event=scheduler_restarting addr=lb:500514852worker2 # [ 83.293683] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:427104853worker2 # [ 83.299206] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:19Z level=info event=worker_gone addr=192.168.1.4:500514854worker1 # [ 83.528465] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500514855worker2 # [ 83.479729] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514856worker1 # [ 83.534047] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4857worker2 # [ 83.485098] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4858worker1 # [ 83.734994] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500514859worker2 # [ 83.685880] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514860worker1 # [ 83.740649] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4861worker2 # [ 83.691184] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4862worker1 # [ 83.941537] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500514863worker2 # [ 83.892050] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514864worker1 # [ 83.946933] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4865worker2 # [ 83.897248] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4866worker1 # [ 84.147840] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500514867worker2 # [ 84.099705] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514868worker1 # [ 84.153354] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4869worker2 # [ 84.104052] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4870worker2 # [ 84.278895] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=scheduler_restarting4871worker1 # [ 84.356413] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500514872worker1 # [ 84.360576] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4873worker2 # [ 84.318780] nix-grpc-daemon[1198]: ts=2026-09-20T09:34:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4874worker2 # [ 84.333648] nix-daemon[832]: reaped child process 1247, status = succeeded4875worker1 # [ 84.557033] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514876worker1 # [ 84.562532] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4877worker1 # [ 84.763427] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514878worker1 # [ 84.769170] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4879worker1 # [ 84.970087] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514880worker1 # [ 84.975564] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4881worker2 # [ 85.093475] nix-grpc-daemon[1198]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4882worker2 # [ 85.093848] nix-grpc-daemon[1198]: E0000 00:00:1789896861.721196 1241 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true4883worker2 # [ 85.094043] nix-grpc-daemon[1198]: E0000 00:00:1789896861.721502 1241 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4884worker1 # [ 85.176310] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500514885worker1 # [ 85.183485] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4886worker1 # [ 85.385657] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500514887worker1 # [ 85.389869] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4888worker1 # [ 85.586540] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500514889worker1 # [ 85.592271] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4890worker1 # [ 85.793222] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500514891worker1 # [ 85.798687] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4892worker1 # [ 85.999546] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500514893worker1 # [ 86.004908] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4894worker2 # [ 86.105008] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4895worker2 # [ 86.108746] systemd[1]: Stopped Nix worker-protocol over gRPC.4896worker2 # [ 86.111196] systemd[1]: nix-grpc-daemon.service: Consumed 514ms CPU time over 10.272s wall clock time, 33.3M memory peak, 21.8K incoming IP traffic, 30.4K outgoing IP traffic.4897worker2 # [ 86.126972] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4898worker2 # [ 86.132549] systemd[1]: Closed Nix worker-protocol over gRPC.4899worker2 # [ 86.135465] systemd[1]: nix-grpc-daemon.socket: Consumed 833us CPU time over 10.410s wall clock time, 256K memory peak, 20.2K incoming IP traffic, 15.6K outgoing IP traffic.4900worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over4901worker1 # [ 86.205916] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500514902worker1 # [ 86.214336] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4903worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)4904worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4905worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.06 seconds)4906worker1 # [ 86.414463] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_connected addr=lb:500514907worker1 # [ 86.418695] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4908worker1 # [ 86.615361] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_connected addr=lb:500514909lb # [ 86.574737] niks3-server[1004]: 2026/09/20 09:34:23 INFO lead: acquired remote=192.168.1.3:341924910worker1 # [ 86.621087] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_take_over4911worker1 # [ 86.624972] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4912worker1 # [ 86.821923] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_connected addr=lb:500514913worker1 # [ 86.827112] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4914worker1 # [ 87.028278] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_connected addr=lb:500514915worker1 # [ 87.033663] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4916worker1 # [ 87.234385] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=scheduler_connected addr=lb:500514917worker1 # [ 87.244668] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04918worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4919worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)4920client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&24921worker1 # [ 87.682148] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447824922worker1 # [ 87.698309] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4923client # don't know how to build these paths:4924client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv4925worker1 # [ 87.719076] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447824926worker1 # [ 87.739710] nix-grpc-daemon[1072]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)4927worker1 # [ 87.767523] nix-grpc-daemon[1072]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 2/5)4928worker1 # [ 87.855772] nix-grpc-daemon[1072]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 204 ms (attempt 3/5)4929worker1 # [ 88.067591] nix-grpc-daemon[1072]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 521 ms (attempt 4/5)4930worker1 # [ 88.597300] nix-grpc-daemon[1072]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4931worker1 # [ 88.630613] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:24Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=04932worker1 # [ 88.638140] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:24Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=14933worker1 # [ 88.677677] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447824934worker1 # [ 88.694833] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447824935client # copying 1 paths to 'lb:50051'...4936client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...4937worker1 # [ 88.709629] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447824938worker1 # [ 88.725521] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4939worker1 # [ 88.752161] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:24Z level=info event=niks3_push_started pid=11464940worker1 # [ 89.133505] nix-daemon[836]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)4941worker1 # [ 89.153633] nix-daemon[836]: reaped child process 1162, status = succeeded4942worker1 # [ 89.331182] nix-daemon[836]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)4943worker1 # [ 89.350042] nix-daemon[836]: reaped child process 1167, status = succeeded4944lb # [ 89.321094] niks3-server[1004]: 2026/09/20 09:34:25 INFO Received uploads request method=POST path=/api/pending_closures4945worker1 # [ 89.374169] nix-grpc-daemon[1146]: time=2026-09-20T09:34:24.990Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4946worker1 # [ 89.375664] nix-grpc-daemon[1146]: time=2026-09-20T09:34:24.990Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"4947lb # [ 89.358603] niks3-server[1004]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst4948worker1 # [ 89.421603] nix-grpc-daemon[1146]: time=2026-09-20T09:34:25.039Z level=INFO msg="Uploading 1 narinfos"4949lb # [ 89.373875] niks3-server[1004]: 2026/09/20 09:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4950lb # [ 89.386552] niks3-server[1004]: 2026/09/20 09:34:26 INFO Signed narinfos id=27 count=14951lb # [ 89.394923] niks3-server[1004]: 2026/09/20 09:34:26 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls4952lb # [ 89.410509] niks3-server[1004]: 2026/09/20 09:34:26 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4953worker1 # [ 89.462191] nix-grpc-daemon[1146]: time=2026-09-20T09:34:25.080Z level=INFO msg="Upload complete. (498ms)"4954lb # [ 89.421157] envoy[902]: conn peer=[2001:db8:1::1]:39354 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=282 ms=16944955lb # [ 89.424867] envoy[902]: conn peer=[2001:db8:1::4]:54834 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3789 tx=1073 ms=82094956lb # [ 89.429610] niks3-server[1004]: 2026/09/20 09:34:26 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo4957lb # [ 89.434136] niks3-server[1004]: 2026/09/20 09:34:26 INFO Completed upload id=274958worker1 # [ 89.472295] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=0 paths=1 nar_bytes_in=5204959worker1 # [ 89.488899] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447824960worker1 # [ 89.497220] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4961worker1 # [ 89.503423] nix-daemon[836]: reaped child process 1144, status = succeeded4962client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...4963worker1 # [ 89.516565] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4964client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv4965client # farm-a> LOG-farm-a-on-standby4966worker1 # [ 90.944458] nix-daemon[836]: reaped child process 1171, status = succeeded4967worker1 # [ 91.106629] nix-daemon[836]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)4968worker1 # [ 91.125244] nix-daemon[836]: reaped child process 1189, status = succeeded4969lb # [ 91.098494] niks3-server[1004]: 2026/09/20 09:34:27 INFO Received uploads request method=POST path=/api/pending_closures4970worker1 # [ 91.152423] nix-grpc-daemon[1146]: time=2026-09-20T09:34:26.769Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4971worker1 # [ 91.153969] nix-grpc-daemon[1146]: time=2026-09-20T09:34:26.769Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"4972lb # [ 91.140603] niks3-server[1004]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv4973lb # [ 91.153384] niks3-server[1004]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst4974worker1 # [ 91.210581] nix-grpc-daemon[1146]: time=2026-09-20T09:34:26.827Z level=INFO msg="Uploading 1 narinfos"4975lb # [ 91.169191] niks3-server[1004]: 2026/09/20 09:34:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4976lb # [ 91.179982] niks3-server[1004]: 2026/09/20 09:34:27 INFO Signed narinfos id=28 count=14977lb # [ 91.182777] niks3-server[1004]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls4978lb # [ 91.191161] niks3-server[1004]: 2026/09/20 09:34:27 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4979worker1 # [ 91.239086] nix-grpc-daemon[1146]: time=2026-09-20T09:34:26.857Z level=INFO msg="Upload complete. (308ms)"4980lb # [ 91.198631] niks3-server[1004]: 2026/09/20 09:34:27 INFO Completed upload id=284981lb # [ 91.201780] niks3-server[1004]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo4982worker1 # [ 91.260223] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=14983worker1 # [ 91.273018] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:26Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=04984worker1 # [ 91.276586] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:26Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=14985worker1 # [ 91.279720] nix-daemon[836]: reaped child process 1169, status = succeeded4986worker1 # [ 91.302782] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447824987worker1 # [ 91.317903] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447824988worker1 # [ 91.334400] nix-grpc-daemon[1146]: time=2026-09-20T09:34:26.952Z level=INFO msg="All 1 paths already cached"4989client # copying 1 paths to 'lb:50051'...4990client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...4991worker1 # [ 91.344878] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447824992worker1 # [ 91.358944] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4993worker1 # [ 91.559624] nix-daemon[836]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)4994worker1 # [ 91.577154] nix-daemon[836]: reaped child process 1198, status = succeeded4995worker1 # [ 91.751792] nix-daemon[836]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)4996worker1 # [ 91.768361] nix-daemon[836]: reaped child process 1203, status = succeeded4997lb # [ 91.740970] niks3-server[1004]: 2026/09/20 09:34:28 INFO Received uploads request method=POST path=/api/pending_closures4998worker1 # [ 91.791801] nix-grpc-daemon[1146]: time=2026-09-20T09:34:27.409Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4999worker1 # [ 91.793469] nix-grpc-daemon[1146]: time=2026-09-20T09:34:27.409Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5000lb # [ 91.769305] niks3-server[1004]: 2026/09/20 09:34:28 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5001worker1 # [ 91.824480] nix-grpc-daemon[1146]: time=2026-09-20T09:34:27.442Z level=INFO msg="Uploading 1 narinfos"5002lb # [ 91.787071] niks3-server[1004]: 2026/09/20 09:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5003lb # [ 91.798002] niks3-server[1004]: 2026/09/20 09:34:28 INFO Signed narinfos id=29 count=15004lb # [ 91.809559] niks3-server[1004]: 2026/09/20 09:34:28 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5005lb # [ 91.813299] niks3-server[1004]: 2026/09/20 09:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5006lb # [ 91.817719] niks3-server[1004]: 2026/09/20 09:34:28 INFO Completed upload id=295007lb # [ 91.820545] niks3-server[1004]: 2026/09/20 09:34:28 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5008worker1 # [ 91.858498] nix-grpc-daemon[1146]: time=2026-09-20T09:34:27.476Z level=INFO msg="Upload complete. (483ms)"5009worker1 # [ 91.869376] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=0 paths=1 nar_bytes_in=6405010worker1 # [ 91.881777] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447825011worker1 # [ 91.887787] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5012worker1 # [ 91.892664] nix-daemon[836]: reaped child process 1193, status = succeeded5013client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5014worker1 # [ 91.904210] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5015client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5016client # farm-b> LOG-farm-b-on-standby5017worker1 # [ 92.750878] nix-daemon[836]: reaped child process 1207, status = succeeded5018worker1 # [ 92.929012] nix-daemon[836]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5019worker1 # [ 92.947031] nix-daemon[836]: reaped child process 1225, status = succeeded5020lb # [ 92.919571] niks3-server[1004]: 2026/09/20 09:34:29 INFO Received uploads request method=POST path=/api/pending_closures5021worker1 # [ 92.971197] nix-grpc-daemon[1146]: time=2026-09-20T09:34:28.588Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5022worker1 # [ 92.972900] nix-grpc-daemon[1146]: time=2026-09-20T09:34:28.588Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5023lb # [ 92.953400] niks3-server[1004]: 2026/09/20 09:34:29 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5024lb # [ 92.966247] niks3-server[1004]: 2026/09/20 09:34:29 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5025lb # [ 92.975014] niks3-server[1004]: 2026/09/20 09:34:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5026worker1 # [ 93.021569] nix-grpc-daemon[1146]: time=2026-09-20T09:34:28.639Z level=INFO msg="Uploading 1 narinfos"5027lb # [ 92.985526] niks3-server[1004]: 2026/09/20 09:34:29 INFO Signed narinfos id=30 count=15028lb # [ 92.991032] niks3-server[1004]: 2026/09/20 09:34:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5029lb # [ 92.999947] niks3-server[1004]: 2026/09/20 09:34:29 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5030lb # [ 93.007779] niks3-server[1004]: 2026/09/20 09:34:29 INFO Completed upload id=305031worker1 # [ 93.048649] nix-grpc-daemon[1146]: time=2026-09-20T09:34:28.666Z level=INFO msg="Upload complete. (333ms)"5032lb # [ 93.011187] niks3-server[1004]: 2026/09/20 09:34:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5033worker1 # [ 93.063649] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15034worker1 # [ 93.084422] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:28Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05035worker1 # [ 93.088982] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:28Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15036worker1 # [ 93.094026] nix-daemon[836]: reaped child process 1205, status = succeeded5037worker1 # [ 93.125658] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447825038worker1 # [ 93.141385] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447825039worker1 # [ 93.156789] nix-grpc-daemon[1146]: time=2026-09-20T09:34:28.774Z level=INFO msg="All 2 paths already cached"5040client # copying 1 paths to 'lb:50051'...5041client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5042worker1 # [ 93.168065] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447825043worker1 # [ 93.181948] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5044worker1 # [ 93.379231] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5045worker1 # [ 93.399670] nix-daemon[836]: reaped child process 1234, status = succeeded5046worker1 # [ 93.573007] nix-daemon[836]: accepted connection from pid 1236, user nix-grpc-daemon (trusted)5047worker1 # [ 93.590704] nix-daemon[836]: reaped child process 1239, status = succeeded5048lb # [ 93.561133] niks3-server[1004]: 2026/09/20 09:34:30 INFO Received uploads request method=POST path=/api/pending_closures5049worker1 # [ 93.613728] nix-grpc-daemon[1146]: time=2026-09-20T09:34:29.231Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5050worker1 # [ 93.615411] nix-grpc-daemon[1146]: time=2026-09-20T09:34:29.231Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5051lb # [ 93.591103] niks3-server[1004]: 2026/09/20 09:34:30 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5052worker1 # [ 93.653145] nix-grpc-daemon[1146]: time=2026-09-20T09:34:29.271Z level=INFO msg="Uploading 1 narinfos"5053lb # [ 93.605313] niks3-server[1004]: 2026/09/20 09:34:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5054lb # [ 93.618345] niks3-server[1004]: 2026/09/20 09:34:30 INFO Signed narinfos id=31 count=15055lb # [ 93.625602] niks3-server[1004]: 2026/09/20 09:34:30 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5056worker1 # [ 93.678133] nix-grpc-daemon[1146]: time=2026-09-20T09:34:29.296Z level=INFO msg="Upload complete. (479ms)"5057lb # [ 93.637162] niks3-server[1004]: 2026/09/20 09:34:30 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5058lb # [ 93.643985] niks3-server[1004]: 2026/09/20 09:34:30 INFO Completed upload id=315059worker1 # [ 93.684277] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=0 paths=1 nar_bytes_in=7685060lb # [ 93.647873] niks3-server[1004]: 2026/09/20 09:34:30 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5061worker1 # [ 93.697986] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447825062worker1 # [ 93.704761] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5063worker1 # [ 93.709623] nix-daemon[836]: reaped child process 1229, status = succeeded5064client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5065client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5066worker1 # [ 93.723437] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5067worker1 # [ 94.948367] nix-daemon[836]: reaped child process 1243, status = succeeded5068client # farm-top> LOG-farm-top-on-standby5069worker1 # [ 95.103745] nix-daemon[836]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5070worker1 # [ 95.123975] nix-daemon[836]: reaped child process 1261, status = succeeded5071lb # [ 95.095292] niks3-server[1004]: 2026/09/20 09:34:31 INFO Received uploads request method=POST path=/api/pending_closures5072worker1 # [ 95.149985] nix-grpc-daemon[1146]: time=2026-09-20T09:34:30.766Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5073worker1 # [ 95.151509] nix-grpc-daemon[1146]: time=2026-09-20T09:34:30.766Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5074lb # [ 95.131458] niks3-server[1004]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5075lb # [ 95.143822] niks3-server[1004]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5076worker1 # [ 95.194505] nix-grpc-daemon[1146]: time=2026-09-20T09:34:30.812Z level=INFO msg="Uploading 1 narinfos"5077lb # [ 95.156696] niks3-server[1004]: 2026/09/20 09:34:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5078lb # [ 95.159926] niks3-server[1004]: 2026/09/20 09:34:31 INFO Signed narinfos id=32 count=15079lb # [ 95.166593] niks3-server[1004]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5080worker1 # [ 95.216827] nix-grpc-daemon[1146]: time=2026-09-20T09:34:30.834Z level=INFO msg="Upload complete. (280ms)"5081lb # [ 95.174886] niks3-server[1004]: 2026/09/20 09:34:31 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5082lb # [ 95.179137] niks3-server[1004]: 2026/09/20 09:34:31 INFO Completed upload id=325083lb # [ 95.181963] niks3-server[1004]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5084worker1 # [ 95.228319] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15085worker1 # [ 95.241211] nix-daemon[836]: reaped child process 1241, status = succeeded5086client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 7.85 seconds)5087client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5088client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5089worker2 # [ 95.548837] systemd[1]: Listening on Nix worker-protocol over gRPC.5090worker2 # [ 95.555647] systemd[1]: Starting Nix worker-protocol over gRPC...5091worker2 # [ 95.749318] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5092worker2 # [ 95.752735] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15093worker2 # [ 95.760415] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5094worker2 # [ 95.762150] systemd[1]: Started Nix worker-protocol over gRPC.5095lb: must succeed: curl -sf localhost:9901/clusters5096worker2 # [ 95.808608] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:32Z level=info event=scheduler_connected addr=lb:500515097worker1 # [ 95.862435] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05098lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5099worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5100worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.08 seconds)5101lb: must succeed: curl -sf localhost:9901/clusters5102lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5103lb: must succeed: curl -sf localhost:9901/clusters5104lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5105worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5106worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.10 seconds)5107(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.42 seconds)5108subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5109client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5110client # [ 96.341775] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5111client # Running as unit: rs.service; invocation ID: fea9bfa20f9b487486b12f704c31ad2a5112client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5113worker2 # [ 96.494490] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500285114worker2 # [ 96.509801] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5115client # [ 96.602040] nix[1150]: don't know how to build these paths:5116client # [ 96.603293] nix[1150]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5117worker1 # [ 96.594094] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:32Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05118worker1 # [ 96.599934] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:32Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15119worker1 # [ 96.638146] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447825120worker1 # [ 96.654605] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447825121client # [ 96.690731] nix[1150]: copying 1 paths to 'lb:50051'...5122client # [ 96.696431] nix[1150]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5123worker1 # [ 96.671981] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447825124worker1 # [ 96.687069] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5125worker1 # [ 96.891137] nix-daemon[836]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5126worker1 # [ 96.911505] nix-daemon[836]: reaped child process 1279, status = succeeded5127worker1 # [ 97.083769] nix-daemon[836]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5128worker1 # [ 97.102987] nix-daemon[836]: reaped child process 1284, status = succeeded5129lb # [ 97.073754] niks3-server[1004]: 2026/09/20 09:34:33 INFO Received uploads request method=POST path=/api/pending_closures5130worker1 # [ 97.124412] nix-grpc-daemon[1146]: time=2026-09-20T09:34:32.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5131worker1 # [ 97.126266] nix-grpc-daemon[1146]: time=2026-09-20T09:34:32.742Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5132lb # [ 97.101710] niks3-server[1004]: 2026/09/20 09:34:33 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5133lb # [ 97.115953] niks3-server[1004]: 2026/09/20 09:34:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5134worker1 # [ 97.163484] nix-grpc-daemon[1146]: time=2026-09-20T09:34:32.781Z level=INFO msg="Uploading 1 narinfos"5135lb # [ 97.129738] niks3-server[1004]: 2026/09/20 09:34:33 INFO Signed narinfos id=33 count=15136lb # [ 97.136722] niks3-server[1004]: 2026/09/20 09:34:33 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5137lb # [ 97.147845] niks3-server[1004]: 2026/09/20 09:34:33 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5138worker1 # [ 97.193509] nix-grpc-daemon[1146]: time=2026-09-20T09:34:32.811Z level=INFO msg="Upload complete. (489ms)"5139lb # [ 97.152992] niks3-server[1004]: 2026/09/20 09:34:33 INFO Completed upload id=335140lb # [ 97.156451] niks3-server[1004]: 2026/09/20 09:34:33 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5141worker1 # [ 97.200133] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=0 paths=1 nar_bytes_in=4165142worker1 # [ 97.208701] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447825143worker1 # [ 97.221286] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5144worker1 # [ 97.227493] nix-daemon[836]: reaped child process 1274, status = succeeded5145worker1 # [ 97.238485] nix-daemon[836]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5146client # [ 97.261291] nix[1150]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5147lb: must succeed: curl -sf localhost:9901/clusters5148lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5149lb: must succeed: curl -sf localhost:9901/clusters5150lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5151lb: must succeed: curl -sf localhost:9901/clusters5152lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5153lb: must succeed: curl -sf localhost:9901/clusters5154lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5155worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5156worker2 # [ 99.219831] systemd-journald[274]: Received client request to rotate journal, rotating.5157worker2 # [ 99.243752] systemd-journald[274]: Vacuuming done, freed 0B of archived journals from /var/log/journal/9a8e8728d2ec43ad861df8acdce5de90.5158worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5159worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5160worker1 # [ 99.354041] systemd[1]: Stopping Nix worker-protocol over gRPC...5161worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5162??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5163 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735164client: must succeed: sleep 5.05165lb # [ 99.401508] envoy[902]: conn peer=[2001:db8:1::1]:41188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9593 tx=3884 ms=75655166worker1 # [ 99.614109] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:35Z level=info event=draining5167client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5168worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5169worker1 # [ 104.893233] nix-daemon[836]: reaped child process 1288, status = succeeded5170worker1 # [ 105.089853] nix-daemon[836]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5171worker1 # [ 105.108550] nix-daemon[836]: reaped child process 1331, status = succeeded5172lb # [ 105.078758] niks3-server[1004]: 2026/09/20 09:34:41 INFO Received uploads request method=POST path=/api/pending_closures5173worker1 # [ 105.136460] nix-grpc-daemon[1146]: time=2026-09-20T09:34:40.751Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5174worker1 # [ 105.138005] nix-grpc-daemon[1146]: time=2026-09-20T09:34:40.751Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5175lb # [ 105.117597] niks3-server[1004]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5176lb # [ 105.130117] niks3-server[1004]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5177lb # [ 105.142070] niks3-server[1004]: 2026/09/20 09:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5178worker1 # [ 105.188883] nix-grpc-daemon[1146]: time=2026-09-20T09:34:40.806Z level=INFO msg="Uploading 1 narinfos"5179lb # [ 105.152175] niks3-server[1004]: 2026/09/20 09:34:41 INFO Signed narinfos id=34 count=15180lb # [ 105.158626] niks3-server[1004]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5181lb # [ 105.170327] niks3-server[1004]: 2026/09/20 09:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5182lb # [ 105.175074] niks3-server[1004]: 2026/09/20 09:34:41 INFO Completed upload id=345183worker1 # [ 105.216605] nix-grpc-daemon[1146]: time=2026-09-20T09:34:40.834Z level=INFO msg="Upload complete. (360ms)"5184lb # [ 105.178702] niks3-server[1004]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5185worker1 # [ 105.233360] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44782 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15186worker1 # [ 105.249672] nix-daemon[836]: reaped child process 1286, status = succeeded5187client # [ 105.295042] systemd[1]: rs.service: Deactivated successfully.5188client # [ 105.299384] systemd[1]: rs.service: Consumed 149ms CPU time over 8.956s wall clock time, 38.9M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5189worker1 # [ 105.615222] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=signal_exit5190worker1 # [ 105.835823] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=scheduler_yield5191lb # [ 105.795537] niks3-server[1004]: 2026/09/20 09:34:42 INFO lead: released remote=192.168.1.3:341925192worker2 # [ 105.789733] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:42Z level=info event=scheduler_restarting addr=lb:500515193worker1 # [ 105.846099] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:447685194worker1 # [ 105.852810] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=worker_gone addr=192.168.1.4:500515195worker1 # [ 105.857056] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=scheduler_restarting addr=lb:500515196worker1 # [ 105.861066] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:447685197worker1 # [ 105.865485] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=worker_gone addr=192.168.1.3:500515198worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.14 seconds)5199client: waiting for success: ! systemctl is-active rs5200client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5201client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5202client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5203client: must fail: journalctl -u rs | grep -q 'reconnecting'5204client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5205lb: must succeed: curl -sf localhost:9901/clusters5206worker2 # [ 105.989897] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:42Z level=info event=scheduler_connected addr=lb:500515207worker2 # [ 105.995486] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5208worker1 # [ 106.053022] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=scheduler_connected addr=lb:500515209worker1 # [ 106.060565] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5210lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5211worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5212worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.08 seconds)5213worker2 # [ 106.196714] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:42Z level=info event=scheduler_connected addr=lb:500515214worker2 # [ 106.201789] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5215worker1 # [ 106.263432] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=scheduler_connected addr=lb:500515216worker1 # [ 106.267708] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5217worker2 # [ 106.402668] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_connected addr=lb:500515218worker2 # [ 106.407830] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5219worker1 # [ 106.463903] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:42Z level=info event=scheduler_connected addr=lb:500515220worker1 # [ 106.469180] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5221worker2 # [ 106.608796] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_connected addr=lb:500515222worker2 # [ 106.613989] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5223worker1 # [ 106.670045] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:42Z level=info event=scheduler_connected addr=lb:500515224worker1 # [ 106.675307] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5225worker1 # [ 106.841925] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:42Z level=info event=scheduler_restarting5226worker2 # [ 106.816767] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_connected addr=lb:500515227worker2 # [ 106.820899] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5228worker1 # [ 106.874350] nix-daemon[836]: reaped child process 1137, status = succeeded5229worker1 # [ 106.899775] nix-grpc-daemon[1072]: ts=2026-09-20T09:34:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5230worker2 # [ 107.017470] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_connected addr=lb:500515231worker2 # [ 107.022653] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5232lb: must succeed: curl -sf localhost:9901/clusters5233worker1 # [ 107.248598] nix-grpc-daemon[1072]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5234worker1 # [ 107.253258] nix-grpc-daemon[1072]: E0000 00:00:1789896882.866522 1115 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5235worker1 # [ 107.259934] nix-grpc-daemon[1072]: E0000 00:00:1789896882.866751 1115 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5236worker2 # [ 107.223410] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_connected addr=lb:500515237worker2 # [ 107.228800] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5238lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5239worker2 # [ 107.429649] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515240worker2 # [ 107.434874] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5241worker2 # [ 107.635648] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515242worker2 # [ 107.640931] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5243worker2 # [ 107.842875] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515244worker2 # [ 107.848151] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5245worker2 # [ 108.048886] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515246worker2 # [ 108.053931] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5247worker1 # [ 108.237165] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5248worker1 # [ 108.243574] systemd[1]: Stopped Nix worker-protocol over gRPC.5249worker1 # [ 108.248824] systemd[1]: nix-grpc-daemon.service: Consumed 1.814s CPU time over 26.846s wall clock time, 36.1M memory peak, 47.6K incoming IP traffic, 99.5K outgoing IP traffic.5250worker1 # [ 108.270284] systemd[1]: Starting Nix worker-protocol over gRPC...5251lb: must succeed: curl -sf localhost:9901/clusters5252worker2 # [ 108.254954] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515253worker2 # [ 108.260340] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5254lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5255worker1 # [ 108.470081] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5256worker1 # [ 108.472695] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15257worker1 # [ 108.480284] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5258worker1 # [ 108.482047] systemd[1]: Started Nix worker-protocol over gRPC.5259worker1 # [ 108.491051] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_take_over5260lb # [ 108.449247] niks3-server[1004]: 2026/09/20 09:34:45 INFO lead: acquired remote=192.168.1.3:360645261worker2 # [ 108.461207] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_connected addr=lb:500515262worker1 # [ 108.518131] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515263worker1 # [ 108.519938] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264worker2 # [ 108.466735] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5265worker2 # [ 108.667644] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_connected addr=lb:500515266worker1 # [ 108.721162] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515267worker2 # [ 108.673268] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5268worker1 # [ 108.726493] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5269worker2 # [ 108.875961] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_connected addr=lb:500515270worker1 # [ 108.927311] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515271worker2 # [ 108.880240] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5272worker1 # [ 108.932901] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5273worker2 # [ 109.076877] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_connected addr=lb:500515274worker1 # [ 109.133720] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515275worker2 # [ 109.082416] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5276worker1 # [ 109.138989] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5277worker2 # [ 109.283244] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:45Z level=info event=scheduler_connected addr=lb:500515278worker1 # [ 109.340303] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=scheduler_connected addr=lb:500515279worker1 # [ 109.346202] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05280worker1 # [ 109.351791] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05281lb: must succeed: curl -sf localhost:9901/clusters5282lb # [ 109.405741] envoy[902]: conn peer=[2001:db8:1::1]:53572 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1326 ms=87005283lb # [ 109.409005] envoy[902]: conn peer=[2001:db8:1::3]:43164 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4430 tx=1400 ms=252425284lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5285worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5286worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.08 seconds)5287(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.29 seconds)5288subtest: scheduler restart mid-build: build finishes, no double build5289client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5290client # [ 109.634152] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5291client # Running as unit: mid.service; invocation ID: 69bc05b36d40431ebffe41a04f01c71c5292client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5293worker2 # [ 109.784893] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500285294client # [ 109.868261] nix[1185]: don't know how to build these paths:5295client # [ 109.869262] nix[1185]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5296worker1 # [ 109.859976] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:45Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05297worker1 # [ 109.865477] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:45Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15298worker1 # [ 109.910135] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387645299worker1 # [ 109.925072] nix-daemon[836]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5300worker1 # [ 109.948391] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)5301worker1 # [ 110.049033] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 2/5)5302worker1 # [ 110.187316] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 255 ms (attempt 3/5)5303worker1 # [ 110.450269] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 537 ms (attempt 4/5)5304worker1 # [ 110.989194] nix-grpc-daemon[1343]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5305worker1 # [ 111.010672] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387645306client # [ 111.049197] nix[1185]: copying 1 paths to 'lb:50051'...5307worker1 # [ 111.030606] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387645308client # [ 111.058548] nix[1185]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5309worker1 # [ 111.050230] nix-daemon[836]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5310worker1 # [ 111.079132] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:46Z level=info event=niks3_push_started pid=14145311worker1 # [ 111.476960] nix-daemon[836]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5312worker1 # [ 111.494874] nix-daemon[836]: reaped child process 1430, status = succeeded5313worker1 # [ 111.673052] nix-daemon[836]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5314worker1 # [ 111.694176] nix-daemon[836]: reaped child process 1435, status = succeeded5315lb # [ 111.664817] niks3-server[1004]: 2026/09/20 09:34:48 INFO Received uploads request method=POST path=/api/pending_closures5316worker1 # [ 111.718956] nix-grpc-daemon[1414]: time=2026-09-20T09:34:47.335Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5317worker1 # [ 111.721332] nix-grpc-daemon[1414]: time=2026-09-20T09:34:47.336Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5318lb # [ 111.709377] niks3-server[1004]: 2026/09/20 09:34:48 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5319worker1 # [ 111.763903] nix-grpc-daemon[1414]: time=2026-09-20T09:34:47.381Z level=INFO msg="Uploading 1 narinfos"5320lb # [ 111.722241] niks3-server[1004]: 2026/09/20 09:34:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5321lb # [ 111.735580] niks3-server[1004]: 2026/09/20 09:34:48 INFO Signed narinfos id=35 count=15322lb # [ 111.742681] niks3-server[1004]: 2026/09/20 09:34:48 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5323lb # [ 111.751144] niks3-server[1004]: 2026/09/20 09:34:48 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5324worker1 # [ 111.801227] nix-grpc-daemon[1414]: time=2026-09-20T09:34:47.419Z level=INFO msg="Upload complete. (498ms)"5325lb # [ 111.761001] niks3-server[1004]: 2026/09/20 09:34:48 INFO Completed upload id=355326worker1 # [ 111.808231] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38764 duration_s=0 paths=1 nar_bytes_in=4245327lb # [ 111.779661] niks3-server[1004]: 2026/09/20 09:34:48 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5328worker1 # [ 111.824631] nix-grpc-daemon[1343]: ts=2026-09-20T09:34:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387645329worker1 # [ 111.832251] nix-daemon[836]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5330worker1 # [ 111.837765] nix-daemon[836]: reaped child process 1412, status = succeeded5331client # [ 111.872243] nix[1185]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5332worker1 # [ 111.852269] nix-daemon[836]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5333lb: must succeed: curl -sf localhost:9901/clusters5334lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5335lb: must succeed: curl -sf localhost:9901/clusters5336lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5337worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5338worker1 # [ 113.624769] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1343 (nix-grpc-daemon) on client request.5339worker1 # [ 113.626420] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5340lb # [ 113.606078] niks3-server[1004]: 2026/09/20 09:34:50 INFO lead: released remote=192.168.1.3:360645341worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.07 seconds)5342client: must succeed: sleep 5.05343client # [ 113.679704] nix[1185]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5344worker2 # [ 113.604299] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5345client # [ 113.686053] nix[1185]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5346worker1 # [ 113.674778] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5347worker1 # [ 113.681672] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5348worker1 # [ 113.682996] systemd[1]: nix-grpc-daemon.service: Consumed 530ms CPU time over 5.402s wall clock time, 33.7M memory peak, 12.3K incoming IP traffic, 16.1K outgoing IP traffic.5349worker1 # [ 113.689860] nix-daemon[836]: reaped child process 1437, status = succeeded5350worker1 # [ 113.690940] nix-daemon[836]: reaped child process 1403, status = succeeded5351worker1 # [ 113.697733] nix-daemon[1439]: unexpected Nix daemon error: error: interrupted by the user5352worker1 # [ 113.705454] nix-daemon[836]: reaped child process 1439, status = failed with exit code 15353worker2 # [ 113.804201] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:50Z level=info event=scheduler_connected addr=lb:500515354worker1 # [ 113.859576] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5355worker1 # [ 113.873277] systemd[1]: Starting Nix worker-protocol over gRPC...5356worker1 # [ 114.087675] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5357worker1 # [ 114.090231] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15358worker1 # [ 114.097842] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5359worker1 # [ 114.099573] systemd[1]: Started Nix worker-protocol over gRPC.5360worker1 # [ 114.111858] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=info event=scheduler_take_over5361lb # [ 114.070025] niks3-server[1004]: 2026/09/20 09:34:50 INFO lead: acquired remote=192.168.1.3:446445362worker2 # [ 114.065370] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5363worker1 # [ 114.139093] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=info event=scheduler_connected addr=lb:500515364worker1 # [ 114.141070] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05365worker1 # [ 114.163164] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05366worker1 # [ 114.165914] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15367worker1 # [ 114.220093] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507245368worker1 # [ 114.231502] nix-daemon[836]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5369worker1 # [ 114.247242] nix-daemon[836]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5370worker1 # [ 114.262274] nix-daemon[836]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5371client # [ 114.285140] nix[1185]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5372worker2 # [ 114.266314] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:50Z level=info event=scheduler_connected addr=lb:500515373worker1 # [ 114.319418] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:49Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05374client: (finished: must succeed: sleep 5.0, in 5.07 seconds)5375client: waiting for success: ! systemctl is-active mid5376worker1 # [ 119.085825] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:54Z level=info event=niks3_push_started pid=15545377worker1 # [ 119.129789] nix-daemon[836]: reaped child process 1530, status = succeeded5378lb # [ 119.405240] envoy[902]: conn peer=[2001:db8:1::3]:47914 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1015 tx=434 ms=51435379worker1 # [ 119.490936] nix-daemon[836]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)5380worker1 # [ 119.510609] nix-daemon[836]: reaped child process 1570, status = succeeded5381lb # [ 119.483967] niks3-server[1004]: 2026/09/20 09:34:56 INFO Received uploads request method=POST path=/api/pending_closures5382worker1 # [ 119.536463] nix-grpc-daemon[1554]: time=2026-09-20T09:34:55.153Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5383worker1 # [ 119.538123] nix-grpc-daemon[1554]: time=2026-09-20T09:34:55.153Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5384lb # [ 119.535751] niks3-server[1004]: 2026/09/20 09:34:56 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5385lb # [ 119.554312] niks3-server[1004]: 2026/09/20 09:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5386worker1 # [ 119.601544] nix-grpc-daemon[1554]: time=2026-09-20T09:34:55.218Z level=INFO msg="Uploading 1 narinfos"5387lb # [ 119.565559] niks3-server[1004]: 2026/09/20 09:34:56 INFO Signed narinfos id=36 count=15388lb # [ 119.567671] niks3-server[1004]: 2026/09/20 09:34:56 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5389lb # [ 119.587123] niks3-server[1004]: 2026/09/20 09:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5390worker1 # [ 119.634484] nix-grpc-daemon[1554]: time=2026-09-20T09:34:55.252Z level=INFO msg="Upload complete. (312ms)"5391lb # [ 119.597968] niks3-server[1004]: 2026/09/20 09:34:56 INFO Completed upload id=365392lb # [ 119.604497] niks3-server[1004]: 2026/09/20 09:34:56 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5393worker1 # [ 119.648348] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50724 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15394lb # [ 119.615472] niks3-server[1004]: 2026/09/20 09:34:56 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5395worker1 # [ 119.664848] nix-daemon[836]: reaped child process 1528, status = succeeded5396client # [ 119.717706] systemd[1]: mid.service: Deactivated successfully.5397client # [ 119.722039] systemd[1]: mid.service: Consumed 166ms CPU time over 10.087s wall clock time, 37M memory peak, 9.5K incoming IP traffic, 10.6K outgoing IP traffic.5398client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5399client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5400client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5401(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.69 seconds)5402subtest: interrupting the client stops the build on the worker5403client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5404client # [ 120.288228] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5405client # Running as unit: intr.service; invocation ID: 2527650d3fde4a0f93e5cf9ee8ac69b25406client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds)5407worker1 # [ 120.481485] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507245408client # [ 120.513853] nix[1220]: don't know how to build these paths:5409client # [ 120.515552] nix[1220]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5410worker1 # [ 120.505125] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:56Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05411worker1 # [ 120.511479] nix-grpc-daemon[1475]: ts=2026-09-20T09:34:56Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15412worker2 # [ 120.504669] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285413worker2 # [ 120.517941] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)5414worker2 # [ 120.574552] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)5415worker2 # [ 120.668876] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 3/5)5416worker2 # [ 120.691350] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 523 ms (attempt 4/5)5417worker2 # [ 121.222615] nix-grpc-daemon[1291]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5418worker2 # [ 121.243142] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500285419client # [ 121.332156] nix[1220]: copying 1 paths to 'lb:50051'...5420worker2 # [ 121.258408] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500285421client # [ 121.335603] nix[1220]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5422worker2 # [ 121.276945] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5423worker2 # [ 121.305081] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:57Z level=info event=niks3_push_started pid=14255424worker2 # [ 121.708086] nix-daemon[832]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5425worker2 # [ 121.725539] nix-daemon[832]: reaped child process 1445, status = succeeded5426worker2 # [ 121.907120] nix-daemon[832]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5427worker2 # [ 121.925065] nix-daemon[832]: reaped child process 1450, status = succeeded5428lb # [ 121.950021] niks3-server[1004]: 2026/09/20 09:34:58 INFO Received uploads request method=POST path=/api/pending_closures5429worker2 # [ 121.955756] nix-grpc-daemon[1425]: time=2026-09-20T09:34:58.582Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5430worker2 # [ 121.957741] nix-grpc-daemon[1425]: time=2026-09-20T09:34:58.582Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5431lb # [ 121.999854] niks3-server[1004]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5432worker2 # [ 122.000419] nix-grpc-daemon[1425]: time=2026-09-20T09:34:58.628Z level=INFO msg="Uploading 1 narinfos"5433lb # [ 122.013538] niks3-server[1004]: 2026/09/20 09:34:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5434lb # [ 122.022923] niks3-server[1004]: 2026/09/20 09:34:58 INFO Signed narinfos id=37 count=15435lb # [ 122.029980] niks3-server[1004]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5436lb # [ 122.038269] niks3-server[1004]: 2026/09/20 09:34:58 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5437lb # [ 122.048604] niks3-server[1004]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5438lb # [ 122.052985] niks3-server[1004]: 2026/09/20 09:34:58 INFO Completed upload id=375439worker2 # [ 122.043381] nix-grpc-daemon[1425]: time=2026-09-20T09:34:58.671Z level=INFO msg="Upload complete. (514ms)"5440worker2 # [ 122.055882] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=0 paths=1 nar_bytes_in=4245441worker2 # [ 122.071920] nix-grpc-daemon[1291]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285442worker2 # [ 122.081168] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5443worker2 # [ 122.084237] nix-daemon[832]: reaped child process 1423, status = succeeded5444client # [ 122.171198] nix[1220]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5445worker2 # [ 122.098411] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5446client: must succeed: systemctl kill -s INT intr5447client # [ 124.023352] systemd[1]: intr.service: Sent signal SIGINT to main process 1220 (nix) on client request.5448client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5449client # [ 124.032709] nix[1220]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5450client # [ 124.037310] nix[1220]: error: interrupted by the user5451client # [ 124.069043] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5452client # [ 124.072066] systemd[1]: intr.service: Failed with result 'exit-code'.5453client # [ 124.074642] systemd[1]: intr.service: Consumed 137ms CPU time over 3.783s wall clock time, 39.2M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.5454worker2 # [ 124.126273] nix-daemon[832]: reaped child process 1452, status = succeeded5455worker2 # [ 124.129454] nix-daemon[1454]: unexpected Nix daemon error: error: interrupted by the user5456worker2 # [ 124.137367] nix-daemon[832]: reaped child process 1454, status = failed with exit code 15457(finished: subtest: interrupting the client stops the build on the worker, in 5.10 seconds)5458subtest: a deploy mid-build drains: the build finishes, the new generation takes over5459client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5460client # [ 125.419853] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5461client # Running as unit: sw.service; invocation ID: bc12e41de87d41de86934901e16374985462client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5463client # [ 125.651325] nix[1239]: don't know how to build these paths:5464client # [ 125.652346] nix[1239]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5465worker2 # [ 125.575660] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500285466worker1 # [ 125.643772] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:01Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05467worker1 # [ 125.649106] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:01Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15468worker1 # [ 125.686181] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507245469worker1 # [ 125.702118] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)5470worker1 # [ 125.742705] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)5471worker1 # [ 125.840295] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 234 ms (attempt 3/5)5472worker1 # [ 126.074793] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 427 ms (attempt 4/5)5473worker1 # [ 126.502624] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5474worker1 # [ 126.524536] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507245475client # [ 126.564232] nix[1239]: copying 1 paths to 'lb:50051'...5476worker1 # [ 126.540545] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507245477client # [ 126.564665] nix[1239]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5478worker1 # [ 126.558198] nix-daemon[836]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5479worker1 # [ 126.777767] nix-daemon[836]: accepted connection from pid 1609, user nix-grpc-daemon (trusted)5480worker1 # [ 126.796357] nix-daemon[836]: reaped child process 1616, status = succeeded5481worker1 # [ 126.970490] nix-daemon[836]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)5482worker1 # [ 126.989642] nix-daemon[836]: reaped child process 1621, status = succeeded5483lb # [ 126.961704] niks3-server[1004]: 2026/09/20 09:35:03 INFO Received uploads request method=POST path=/api/pending_closures5484worker1 # [ 127.013924] nix-grpc-daemon[1554]: time=2026-09-20T09:35:02.631Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5485worker1 # [ 127.015512] nix-grpc-daemon[1554]: time=2026-09-20T09:35:02.631Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5486lb # [ 126.993735] niks3-server[1004]: 2026/09/20 09:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5487worker1 # [ 127.044675] nix-grpc-daemon[1554]: time=2026-09-20T09:35:02.662Z level=INFO msg="Uploading 1 narinfos"5488lb # [ 127.007369] niks3-server[1004]: 2026/09/20 09:35:03 INFO Signed narinfos id=38 count=15489lb # [ 127.015410] niks3-server[1004]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5490lb # [ 127.025509] niks3-server[1004]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5491lb # [ 127.034457] niks3-server[1004]: 2026/09/20 09:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5492worker1 # [ 127.080820] nix-grpc-daemon[1554]: time=2026-09-20T09:35:02.699Z level=INFO msg="Upload complete. (505ms)"5493lb # [ 127.041296] niks3-server[1004]: 2026/09/20 09:35:03 INFO Completed upload id=385494lb # [ 127.044544] niks3-server[1004]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5495worker1 # [ 127.088804] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50724 duration_s=0 paths=1 nar_bytes_in=4165496worker1 # [ 127.101895] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507245497worker1 # [ 127.108504] nix-daemon[836]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5498worker1 # [ 127.114059] nix-daemon[836]: reaped child process 1607, status = succeeded5499client # [ 127.148811] nix[1239]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5500worker1 # [ 127.127469] nix-daemon[836]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5501worker1: must succeed: systemctl reload nix-grpc-daemon.service5502worker1 # [ 129.338757] systemd[1]: Reloading Nix worker-protocol over gRPC...5503worker1 # [ 129.390400] systemd[1]: Reloaded Nix worker-protocol over gRPC.5504worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.11 seconds)5505worker1: must succeed: pgrep -f 'read -t [9]0 x'5506lb # [ 129.403444] envoy[902]: conn peer=[2001:db8:1::1]:55412 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3854 tx=1658 ms=98305507worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5508client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25509lb # [ 129.408541] envoy[902]: conn peer=[2001:db8:1::1]:36456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=757 ms=35445510worker2 # [ 129.651800] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500285511client # don't know how to build these paths:5512client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5513worker2 # [ 129.660861] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500285514worker1 # [ 129.735968] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:05Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05515worker1 # [ 129.738047] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:05Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15516worker2 # [ 129.735234] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285517worker2 # [ 129.750390] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500285518client # copying 1 paths to 'lb:50051'...5519worker2 # [ 129.766374] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500285520client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5521worker2 # [ 129.791766] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5522worker2 # [ 130.009573] nix-daemon[832]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)5523worker2 # [ 130.028943] nix-daemon[832]: reaped child process 1511, status = succeeded5524worker1 # [ 130.114543] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:05Z level=info event=draining5525worker2 # [ 130.212972] nix-daemon[832]: accepted connection from pid 1513, user nix-grpc-daemon (trusted)5526worker2 # [ 130.231643] nix-daemon[832]: reaped child process 1516, status = succeeded5527lb # [ 130.254269] niks3-server[1004]: 2026/09/20 09:35:06 INFO Received uploads request method=POST path=/api/pending_closures5528worker2 # [ 130.254542] nix-grpc-daemon[1425]: time=2026-09-20T09:35:06.881Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5529worker2 # [ 130.256072] nix-grpc-daemon[1425]: time=2026-09-20T09:35:06.881Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5530lb # [ 130.283647] niks3-server[1004]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5531worker2 # [ 130.295161] nix-grpc-daemon[1425]: time=2026-09-20T09:35:06.922Z level=INFO msg="Uploading 1 narinfos"5532lb # [ 130.296884] niks3-server[1004]: 2026/09/20 09:35:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5533lb # [ 130.312676] niks3-server[1004]: 2026/09/20 09:35:06 INFO Signed narinfos id=39 count=15534lb # [ 130.320189] niks3-server[1004]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5535worker2 # [ 130.320116] nix-grpc-daemon[1425]: time=2026-09-20T09:35:06.948Z level=INFO msg="Upload complete. (510ms)"5536lb # [ 130.332046] niks3-server[1004]: 2026/09/20 09:35:06 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5537lb # [ 130.338283] niks3-server[1004]: 2026/09/20 09:35:06 INFO Completed upload id=395538worker2 # [ 130.328275] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=0 paths=1 nar_bytes_in=5285539lb # [ 130.342235] niks3-server[1004]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5540worker2 # [ 130.336667] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285541worker2 # [ 130.347356] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5542worker2 # [ 130.352411] nix-daemon[832]: reaped child process 1506, status = succeeded5543client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5544client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5545worker2 # [ 130.366492] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5546worker2 # [ 131.189063] nix-daemon[832]: reaped child process 1520, status = succeeded5547client # farm-a> LOG-farm-a-during-drain5548worker2 # [ 131.353638] nix-daemon[832]: accepted connection from pid 1535, user nix-grpc-daemon (trusted)5549worker2 # [ 131.372771] nix-daemon[832]: reaped child process 1538, status = succeeded5550lb # [ 131.397940] niks3-server[1004]: 2026/09/20 09:35:08 INFO Received uploads request method=POST path=/api/pending_closures5551worker2 # [ 131.400275] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.025Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5552worker2 # [ 131.401749] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.025Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5553lb # [ 131.432700] niks3-server[1004]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5554lb # [ 131.446110] niks3-server[1004]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5555lb # [ 131.460504] niks3-server[1004]: 2026/09/20 09:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5556worker2 # [ 131.457662] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.084Z level=INFO msg="Uploading 1 narinfos"5557lb # [ 131.470761] niks3-server[1004]: 2026/09/20 09:35:08 INFO Signed narinfos id=40 count=15558lb # [ 131.477436] niks3-server[1004]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5559lb # [ 131.486628] niks3-server[1004]: 2026/09/20 09:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5560worker2 # [ 131.482494] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.110Z level=INFO msg="Upload complete. (306ms)"5561lb # [ 131.493510] niks3-server[1004]: 2026/09/20 09:35:08 INFO Completed upload id=405562lb # [ 131.496526] niks3-server[1004]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5563worker2 # [ 131.492958] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15564worker1 # [ 131.549326] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:07Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05565worker1 # [ 131.551169] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:07Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15566worker2 # [ 131.509184] nix-daemon[832]: reaped child process 1518, status = succeeded5567worker2 # [ 131.547792] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285568worker2 # [ 131.564144] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500285569worker2 # [ 131.579082] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.207Z level=INFO msg="All 1 paths already cached"5570client # copying 1 paths to 'lb:50051'...5571worker2 # [ 131.589469] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500285572client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5573worker2 # [ 131.612262] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5574worker2 # [ 131.827701] nix-daemon[832]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5575worker2 # [ 131.845325] nix-daemon[832]: reaped child process 1545, status = succeeded5576worker2 # [ 132.027978] nix-daemon[832]: accepted connection from pid 1547, user nix-grpc-daemon (trusted)5577worker2 # [ 132.044918] nix-daemon[832]: reaped child process 1550, status = succeeded5578lb # [ 132.068482] niks3-server[1004]: 2026/09/20 09:35:08 INFO Received uploads request method=POST path=/api/pending_closures5579worker2 # [ 132.068358] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.695Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5580worker2 # [ 132.069825] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.696Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5581lb # [ 132.096962] niks3-server[1004]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5582lb # [ 132.110817] niks3-server[1004]: 2026/09/20 09:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5583worker2 # [ 132.107097] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.734Z level=INFO msg="Uploading 1 narinfos"5584lb # [ 132.123575] niks3-server[1004]: 2026/09/20 09:35:08 INFO Signed narinfos id=41 count=15585lb # [ 132.129258] niks3-server[1004]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5586worker2 # [ 132.128029] nix-grpc-daemon[1425]: time=2026-09-20T09:35:08.756Z level=INFO msg="Upload complete. (496ms)"5587lb # [ 132.138701] niks3-server[1004]: 2026/09/20 09:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5588lb # [ 132.144089] niks3-server[1004]: 2026/09/20 09:35:08 INFO Completed upload id=415589lb # [ 132.147429] niks3-server[1004]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5590worker2 # [ 132.133923] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=0 paths=1 nar_bytes_in=6405591worker2 # [ 132.146867] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285592worker2 # [ 132.153367] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5593worker2 # [ 132.157898] nix-daemon[832]: reaped child process 1540, status = succeeded5594client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5595client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5596worker2 # [ 132.171132] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5597client # farm-b> LOG-farm-b-during-drain5598worker2 # [ 133.601263] nix-daemon[832]: reaped child process 1554, status = succeeded5599worker2 # [ 133.766714] nix-daemon[832]: accepted connection from pid 1569, user nix-grpc-daemon (trusted)5600worker2 # [ 133.786026] nix-daemon[832]: reaped child process 1572, status = succeeded5601lb # [ 133.811242] niks3-server[1004]: 2026/09/20 09:35:10 INFO Received uploads request method=POST path=/api/pending_closures5602worker2 # [ 133.810853] nix-grpc-daemon[1425]: time=2026-09-20T09:35:10.437Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5603worker2 # [ 133.812449] nix-grpc-daemon[1425]: time=2026-09-20T09:35:10.438Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5604lb # [ 133.841375] niks3-server[1004]: 2026/09/20 09:35:10 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5605lb # [ 133.849592] niks3-server[1004]: 2026/09/20 09:35:10 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5606lb # [ 133.860138] niks3-server[1004]: 2026/09/20 09:35:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5607lb # [ 133.866157] niks3-server[1004]: 2026/09/20 09:35:10 INFO Signed narinfos id=42 count=15608worker2 # [ 133.856679] nix-grpc-daemon[1425]: time=2026-09-20T09:35:10.484Z level=INFO msg="Uploading 1 narinfos"5609lb # [ 133.871331] niks3-server[1004]: 2026/09/20 09:35:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5610lb # [ 133.880166] niks3-server[1004]: 2026/09/20 09:35:10 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5611lb # [ 133.886224] niks3-server[1004]: 2026/09/20 09:35:10 INFO Completed upload id=425612worker2 # [ 133.875852] nix-grpc-daemon[1425]: time=2026-09-20T09:35:10.503Z level=INFO msg="Upload complete. (294ms)"5613lb # [ 133.888213] niks3-server[1004]: 2026/09/20 09:35:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5614worker1 # [ 133.938789] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:09Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05615worker1 # [ 133.940766] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:09Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15616worker2 # [ 133.888967] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15617worker2 # [ 133.902572] nix-daemon[832]: reaped child process 1552, status = succeeded5618worker2 # [ 133.938468] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285619worker2 # [ 133.961161] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500285620worker2 # [ 133.975554] nix-grpc-daemon[1425]: time=2026-09-20T09:35:10.603Z level=INFO msg="All 2 paths already cached"5621client # copying 1 paths to 'lb:50051'...5622worker2 # [ 133.986152] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500285623client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5624worker2 # [ 134.008253] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5625worker2 # [ 134.220076] nix-daemon[832]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)5626worker2 # [ 134.238006] nix-daemon[832]: reaped child process 1579, status = succeeded5627worker2 # [ 134.416402] nix-daemon[832]: accepted connection from pid 1581, user nix-grpc-daemon (trusted)5628worker2 # [ 134.433427] nix-daemon[832]: reaped child process 1584, status = succeeded5629lb # [ 134.456271] niks3-server[1004]: 2026/09/20 09:35:11 INFO Received uploads request method=POST path=/api/pending_closures5630worker2 # [ 134.455870] nix-grpc-daemon[1425]: time=2026-09-20T09:35:11.083Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5631worker2 # [ 134.457580] nix-grpc-daemon[1425]: time=2026-09-20T09:35:11.083Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5632lb # [ 134.483564] niks3-server[1004]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5633lb # [ 134.496546] niks3-server[1004]: 2026/09/20 09:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5634worker2 # [ 134.492373] nix-grpc-daemon[1425]: time=2026-09-20T09:35:11.120Z level=INFO msg="Uploading 1 narinfos"5635lb # [ 134.508124] niks3-server[1004]: 2026/09/20 09:35:11 INFO Signed narinfos id=43 count=15636lb # [ 134.513334] niks3-server[1004]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5637lb # [ 134.520082] niks3-server[1004]: 2026/09/20 09:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5638lb # [ 134.527472] niks3-server[1004]: 2026/09/20 09:35:11 INFO Completed upload id=435639worker2 # [ 134.517539] nix-grpc-daemon[1425]: time=2026-09-20T09:35:11.145Z level=INFO msg="Upload complete. (490ms)"5640lb # [ 134.530881] niks3-server[1004]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5641worker2 # [ 134.524134] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=0 paths=1 nar_bytes_in=7685642worker2 # [ 134.532701] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500285643worker2 # [ 134.542752] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5644worker2 # [ 134.547229] nix-daemon[832]: reaped child process 1574, status = succeeded5645client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5646worker2 # [ 134.560617] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5647client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5648client # farm-top> LOG-farm-top-during-drain5649worker2 # [ 135.407722] nix-daemon[832]: reaped child process 1588, status = succeeded5650worker2 # [ 135.585538] nix-daemon[832]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)5651worker2 # [ 135.603359] nix-daemon[832]: reaped child process 1606, status = succeeded5652lb # [ 135.626171] niks3-server[1004]: 2026/09/20 09:35:12 INFO Received uploads request method=POST path=/api/pending_closures5653worker2 # [ 135.627422] nix-grpc-daemon[1425]: time=2026-09-20T09:35:12.253Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5654worker2 # [ 135.629118] nix-grpc-daemon[1425]: time=2026-09-20T09:35:12.253Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5655lb # [ 135.656863] niks3-server[1004]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5656lb # [ 135.670355] niks3-server[1004]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5657worker2 # [ 135.673934] nix-grpc-daemon[1425]: time=2026-09-20T09:35:12.301Z level=INFO msg="Uploading 1 narinfos"5658lb # [ 135.684458] niks3-server[1004]: 2026/09/20 09:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5659lb # [ 135.693356] niks3-server[1004]: 2026/09/20 09:35:12 INFO Signed narinfos id=44 count=15660lb # [ 135.698002] niks3-server[1004]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5661lb # [ 135.705237] niks3-server[1004]: 2026/09/20 09:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5662lb # [ 135.709818] niks3-server[1004]: 2026/09/20 09:35:12 INFO Completed upload id=445663worker2 # [ 135.699952] nix-grpc-daemon[1425]: time=2026-09-20T09:35:12.327Z level=INFO msg="Upload complete. (329ms)"5664lb # [ 135.712188] niks3-server[1004]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5665worker2 # [ 135.713055] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15666worker2 # [ 135.727285] nix-daemon[832]: reaped child process 1586, status = succeeded5667client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.36 seconds)5668client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5669client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.24 seconds)5670worker1: must succeed: pkill -f 'read -t [9]0 x'5671worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5672client: waiting for success: ! systemctl is-active sw5673worker1 # [ 136.157872] nix-daemon[836]: reaped child process 1625, status = succeeded5674worker1 # [ 136.319563] nix-daemon[836]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)5675worker1 # [ 136.340118] nix-daemon[836]: reaped child process 1674, status = succeeded5676lb # [ 136.310111] niks3-server[1004]: 2026/09/20 09:35:12 INFO Received uploads request method=POST path=/api/pending_closures5677worker1 # [ 136.363489] nix-grpc-daemon[1554]: time=2026-09-20T09:35:11.980Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5678worker1 # [ 136.365174] nix-grpc-daemon[1554]: time=2026-09-20T09:35:11.980Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5679lb # [ 136.344372] niks3-server[1004]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5680lb # [ 136.357483] niks3-server[1004]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5681lb # [ 136.371716] niks3-server[1004]: 2026/09/20 09:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5682worker1 # [ 136.418794] nix-grpc-daemon[1554]: time=2026-09-20T09:35:12.035Z level=INFO msg="Uploading 1 narinfos"5683lb # [ 136.380785] niks3-server[1004]: 2026/09/20 09:35:13 INFO Signed narinfos id=45 count=15684lb # [ 136.383325] niks3-server[1004]: 2026/09/20 09:35:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5685lb # [ 136.394100] niks3-server[1004]: 2026/09/20 09:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5686lb # [ 136.401257] niks3-server[1004]: 2026/09/20 09:35:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5687lb # [ 136.405763] niks3-server[1004]: 2026/09/20 09:35:13 INFO Completed upload id=455688worker1 # [ 136.446872] nix-grpc-daemon[1554]: time=2026-09-20T09:35:12.065Z level=INFO msg="Upload complete. (301ms)"5689worker1 # [ 136.461118] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50724 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15690worker1 # [ 136.475270] nix-daemon[836]: reaped child process 1623, status = succeeded5691client # [ 136.524675] systemd[1]: sw.service: Deactivated successfully.5692client # [ 136.530322] systemd[1]: sw.service: Consumed 156ms CPU time over 11.108s wall clock time, 38.9M memory peak, 8.8K incoming IP traffic, 9K outgoing IP traffic.5693worker1 # [ 137.118508] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=signal_exit5694client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)5695client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5696client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5697worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5698worker1 # [ 137.331045] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=scheduler_yield5699lb # [ 137.294146] niks3-server[1004]: 2026/09/20 09:35:13 INFO lead: released remote=192.168.1.3:446445700worker2 # [ 137.287343] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:13Z level=info event=scheduler_restarting addr=lb:500515701worker1 # [ 137.341939] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:507225702worker1 # [ 137.346971] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=worker_gone addr=192.168.1.4:500515703worker1 # [ 137.358538] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=scheduler_restarting addr=lb:500515704worker1 # [ 137.364116] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:507225705worker1 # [ 137.370335] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:12Z level=info event=worker_gone addr=192.168.1.3:500515706worker2 # [ 137.489511] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_connected addr=lb:500515707worker2 # [ 137.496079] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5708worker1 # [ 137.553214] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_connected addr=lb:500515709worker1 # [ 137.557973] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5710worker2 # [ 137.697269] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_connected addr=lb:500515711worker1 # [ 137.753380] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_connected addr=lb:500515712worker2 # [ 137.703194] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5713worker1 # [ 137.758958] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5714worker2 # [ 137.904255] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_connected addr=lb:500515715worker1 # [ 137.961041] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_connected addr=lb:500515716worker2 # [ 137.909914] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5717worker1 # [ 137.966656] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5718worker2 # [ 138.111022] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_connected addr=lb:500515719worker1 # [ 138.168787] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_connected addr=lb:500515720worker2 # [ 138.117007] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5721worker1 # [ 138.173200] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5722worker1 # [ 138.332391] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:13Z level=info event=scheduler_restarting5723worker2 # [ 138.323543] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_connected addr=lb:500515724worker2 # [ 138.328123] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5725worker1 # [ 138.402785] nix-grpc-daemon[1475]: ts=2026-09-20T09:35:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5726worker1 # [ 138.407922] nix-daemon[836]: reaped child process 1526, status = succeeded5727worker2 # [ 138.523199] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_connected addr=lb:500515728worker2 # [ 138.529107] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5729worker2 # [ 138.730613] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_connected addr=lb:500515730worker2 # [ 138.736568] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5731worker2 # [ 138.938793] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_connected addr=lb:500515732worker2 # [ 138.944879] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5733worker2 # [ 139.145953] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_connected addr=lb:500515734worker2 # [ 139.151817] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5735worker1 # [ 139.305204] nix-grpc-daemon[1475]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5736worker1 # [ 139.305655] nix-grpc-daemon[1475]: E0000 00:00:1789896914.921528 1518 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5737worker1 # [ 139.305810] nix-grpc-daemon[1475]: E0000 00:00:1789896914.921916 1518 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5738worker2 # [ 139.354716] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_connected addr=lb:500515739worker2 # [ 139.360659] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5740lb # [ 139.405419] envoy[902]: conn peer=[2001:db8:1::1]:51642 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9556 tx=3830 ms=60745741lb # [ 139.411023] envoy[902]: conn peer=[2001:db8:1::1]:51626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1351 ms=108465742lb # [ 139.416241] envoy[902]: conn peer=[2001:db8:1::3]:55506 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1426 tx=539 ms=242405743worker2 # [ 139.562952] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515744worker2 # [ 139.569327] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5745worker2 # [ 139.770145] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515746worker2 # [ 139.776113] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5747worker2 # [ 139.977964] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515748worker2 # [ 139.983601] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5749worker1 # [ 140.162944] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5750worker1 # [ 140.166764] systemd[1]: nix-grpc-daemon.service: Consumed 895ms CPU time over 26.484s wall clock time, 34.6M memory peak, 23.7K incoming IP traffic, 32.3K outgoing IP traffic.5751worker1 # [ 140.186104] systemd[1]: Starting Nix worker-protocol over gRPC...5752worker2 # [ 140.184720] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515753worker2 # [ 140.190812] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker1 # [ 140.405221] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5755worker1 # [ 140.412845] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15756worker1 # [ 140.422179] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5757worker1 # [ 140.423959] systemd[1]: Started Nix worker-protocol over gRPC.5758lb # [ 140.397849] niks3-server[1004]: 2026/09/20 09:35:17 INFO lead: acquired remote=192.168.1.3:462825759worker1 # [ 140.448558] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_take_over5760worker2 # [ 140.394033] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_connected addr=lb:500515761worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.17 seconds)5762worker2 # [ 140.399933] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5763lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5764lb: must succeed: curl -sf localhost:9901/clusters5765worker1 # [ 140.467847] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515766worker1 # [ 140.470147] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5767lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5768worker2 # [ 140.601120] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_connected addr=lb:500515769worker2 # [ 140.607167] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5770worker1 # [ 140.672257] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515771worker1 # [ 140.677993] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5772worker2 # [ 140.808165] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_connected addr=lb:500515773worker2 # [ 140.813836] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5774worker1 # [ 140.879090] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515775worker1 # [ 140.885006] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5776worker2 # [ 141.014606] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_connected addr=lb:500515777worker2 # [ 141.020338] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5778worker1 # [ 141.086531] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515779worker1 # [ 141.091857] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5780worker2 # [ 141.220908] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:17Z level=info event=scheduler_connected addr=lb:500515781worker1 # [ 141.277944] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05782worker1 # [ 141.288506] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=scheduler_connected addr=lb:500515783worker1 # [ 141.293927] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05784lb: must succeed: curl -sf localhost:9901/clusters5785lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5786lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.18 seconds)5787lb: must succeed: curl -sf localhost:9901/clusters5788lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5789worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5790worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)5791(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.46 seconds)5792subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5793client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15794worker1 # [ 142.057295] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:456765795worker1 # [ 142.072703] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5796worker1 # [ 142.093143] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456765797worker1 # [ 142.113328] nix-grpc-daemon[1688]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)5798worker1 # [ 142.188064] nix-grpc-daemon[1688]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 2/5)5799worker1 # [ 142.196975] nix-grpc-daemon[1688]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 3/5)5800worker1 # [ 142.204366] nix-grpc-daemon[1688]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 740 ms (attempt 4/5)5801worker1 # [ 142.937602] nix-grpc-daemon[1688]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5802worker1 # [ 142.991705] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05803worker1 # [ 142.999085] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15804worker1 # [ 143.005226] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05805worker1 # [ 143.010438] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05806worker1 # [ 143.040882] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765807worker1 # [ 143.057813] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456765808worker1 # [ 143.072129] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456765809worker1 # [ 143.087638] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5810worker1 # [ 143.115198] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:18Z level=info event=niks3_push_started pid=17515811worker1 # [ 143.517468] nix-daemon[836]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)5812worker1 # [ 143.536622] nix-daemon[836]: reaped child process 1767, status = succeeded5813worker1 # [ 143.712424] nix-daemon[836]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)5814worker1 # [ 143.729401] nix-daemon[836]: reaped child process 1772, status = succeeded5815lb # [ 143.701889] niks3-server[1004]: 2026/09/20 09:35:20 INFO Received uploads request method=POST path=/api/pending_closures5816worker1 # [ 143.756266] nix-grpc-daemon[1751]: time=2026-09-20T09:35:19.372Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5817worker1 # [ 143.757759] nix-grpc-daemon[1751]: time=2026-09-20T09:35:19.372Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5818lb # [ 143.744172] niks3-server[1004]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5819worker1 # [ 143.799795] nix-grpc-daemon[1751]: time=2026-09-20T09:35:19.417Z level=INFO msg="Uploading 1 narinfos"5820lb # [ 143.757938] niks3-server[1004]: 2026/09/20 09:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5821lb # [ 143.769033] niks3-server[1004]: 2026/09/20 09:35:20 INFO Signed narinfos id=46 count=15822lb # [ 143.775609] niks3-server[1004]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5823lb # [ 143.785465] niks3-server[1004]: 2026/09/20 09:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5824worker1 # [ 143.836008] nix-grpc-daemon[1751]: time=2026-09-20T09:35:19.454Z level=INFO msg="Upload complete. (493ms)"5825lb # [ 143.796336] niks3-server[1004]: 2026/09/20 09:35:20 INFO Completed upload id=465826lb # [ 143.800544] niks3-server[1004]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5827worker1 # [ 143.844160] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=5125828worker1 # [ 143.857008] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765829worker1 # [ 143.868105] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5830worker1 # [ 143.873369] nix-daemon[836]: reaped child process 1749, status = succeeded5831worker1 # [ 143.887195] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5832worker1 # [ 145.729677] nix-daemon[836]: reaped child process 1776, status = succeeded5833worker1 # [ 145.915147] nix-daemon[836]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)5834worker1 # [ 145.932672] nix-daemon[836]: reaped child process 1794, status = succeeded5835lb # [ 145.903394] niks3-server[1004]: 2026/09/20 09:35:22 INFO Received uploads request method=POST path=/api/pending_closures5836worker1 # [ 145.956420] nix-grpc-daemon[1751]: time=2026-09-20T09:35:21.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5837worker1 # [ 145.957994] nix-grpc-daemon[1751]: time=2026-09-20T09:35:21.571Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5838lb # [ 145.936390] niks3-server[1004]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5839lb # [ 145.949667] niks3-server[1004]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5840lb # [ 145.962537] niks3-server[1004]: 2026/09/20 09:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5841lb # [ 145.967921] niks3-server[1004]: 2026/09/20 09:35:22 INFO Signed narinfos id=47 count=15842worker1 # [ 146.009532] nix-grpc-daemon[1751]: time=2026-09-20T09:35:21.626Z level=INFO msg="Uploading 1 narinfos"5843lb # [ 145.972234] niks3-server[1004]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5844lb # [ 145.983520] niks3-server[1004]: 2026/09/20 09:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5845worker1 # [ 146.029959] nix-grpc-daemon[1751]: time=2026-09-20T09:35:21.648Z level=INFO msg="Upload complete. (330ms)"5846lb # [ 145.992256] niks3-server[1004]: 2026/09/20 09:35:22 INFO Completed upload id=475847lb # [ 146.001709] niks3-server[1004]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5848worker1 # [ 146.048970] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=15849worker1 # [ 146.064813] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:21Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15850worker1 # [ 146.069800] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:21Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05851worker1 # [ 146.073552] nix-daemon[836]: reaped child process 1774, status = succeeded5852worker1 # [ 146.092135] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765853worker1 # [ 146.103034] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456765854worker1 # [ 146.116128] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456765855worker1 # [ 146.130278] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5856worker1 # [ 146.325810] nix-daemon[836]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)5857worker1 # [ 146.344866] nix-daemon[836]: reaped child process 1803, status = succeeded5858worker1 # [ 146.521804] nix-daemon[836]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)5859worker1 # [ 146.539397] nix-daemon[836]: reaped child process 1808, status = succeeded5860lb # [ 146.508989] niks3-server[1004]: 2026/09/20 09:35:23 INFO Received uploads request method=POST path=/api/pending_closures5861worker1 # [ 146.563989] nix-grpc-daemon[1751]: time=2026-09-20T09:35:22.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5862worker1 # [ 146.565504] nix-grpc-daemon[1751]: time=2026-09-20T09:35:22.181Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5863lb # [ 146.542220] niks3-server[1004]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5864worker1 # [ 146.596509] nix-grpc-daemon[1751]: time=2026-09-20T09:35:22.214Z level=INFO msg="Uploading 1 narinfos"5865lb # [ 146.560189] niks3-server[1004]: 2026/09/20 09:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5866lb # [ 146.569419] niks3-server[1004]: 2026/09/20 09:35:23 INFO Signed narinfos id=48 count=15867worker1 # [ 146.619882] nix-grpc-daemon[1751]: time=2026-09-20T09:35:22.238Z level=INFO msg="Upload complete. (475ms)"5868lb # [ 146.579375] niks3-server[1004]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5869lb # [ 146.584405] niks3-server[1004]: 2026/09/20 09:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5870worker1 # [ 146.628282] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=5125871lb # [ 146.589795] niks3-server[1004]: 2026/09/20 09:35:23 INFO Completed upload id=485872lb # [ 146.592199] niks3-server[1004]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5873worker1 # [ 146.641521] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765874worker1 # [ 146.648100] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5875worker1 # [ 146.653177] nix-daemon[836]: reaped child process 1798, status = succeeded5876worker1 # [ 146.666991] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5877worker1 # [ 147.905043] nix-daemon[836]: reaped child process 1812, status = succeeded5878worker1 # [ 148.070838] nix-daemon[836]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)5879worker1 # [ 148.091468] nix-daemon[836]: reaped child process 1830, status = succeeded5880lb # [ 148.063103] niks3-server[1004]: 2026/09/20 09:35:24 INFO Received uploads request method=POST path=/api/pending_closures5881worker1 # [ 148.115355] nix-grpc-daemon[1751]: time=2026-09-20T09:35:23.732Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5882worker1 # [ 148.117089] nix-grpc-daemon[1751]: time=2026-09-20T09:35:23.732Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5883lb # [ 148.099905] niks3-server[1004]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5884lb # [ 148.113060] niks3-server[1004]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5885lb # [ 148.122241] niks3-server[1004]: 2026/09/20 09:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5886worker1 # [ 148.170110] nix-grpc-daemon[1751]: time=2026-09-20T09:35:23.787Z level=INFO msg="Uploading 1 narinfos"5887lb # [ 148.134267] niks3-server[1004]: 2026/09/20 09:35:24 INFO Signed narinfos id=49 count=15888lb # [ 148.141671] niks3-server[1004]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5889lb # [ 148.148234] niks3-server[1004]: 2026/09/20 09:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5890lb # [ 148.155975] niks3-server[1004]: 2026/09/20 09:35:24 INFO Completed upload id=495891worker1 # [ 148.197480] nix-grpc-daemon[1751]: time=2026-09-20T09:35:23.815Z level=INFO msg="Upload complete. (319ms)"5892lb # [ 148.158514] niks3-server[1004]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5893worker1 # [ 148.215743] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=15894worker1 # [ 148.232253] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:23Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15895worker1 # [ 148.242106] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765896worker1 # [ 148.245974] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:23Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05897worker1 # [ 148.250749] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456765898worker1 # [ 148.260760] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456765899worker1 # [ 148.265643] nix-daemon[836]: reaped child process 1810, status = succeeded5900worker1 # [ 148.267367] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5901worker1 # [ 148.453801] nix-daemon[836]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)5902worker1 # [ 148.473350] nix-daemon[836]: reaped child process 1839, status = succeeded5903worker1 # [ 148.645520] nix-daemon[836]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)5904worker1 # [ 148.665489] nix-daemon[836]: reaped child process 1844, status = succeeded5905lb # [ 148.636705] niks3-server[1004]: 2026/09/20 09:35:25 INFO Received uploads request method=POST path=/api/pending_closures5906worker1 # [ 148.686885] nix-grpc-daemon[1751]: time=2026-09-20T09:35:24.304Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5907worker1 # [ 148.688471] nix-grpc-daemon[1751]: time=2026-09-20T09:35:24.304Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5908lb # [ 148.663683] niks3-server[1004]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5909lb # [ 148.676808] niks3-server[1004]: 2026/09/20 09:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5910worker1 # [ 148.724275] nix-grpc-daemon[1751]: time=2026-09-20T09:35:24.342Z level=INFO msg="Uploading 1 narinfos"5911lb # [ 148.688973] niks3-server[1004]: 2026/09/20 09:35:25 INFO Signed narinfos id=50 count=15912lb # [ 148.694725] niks3-server[1004]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5913lb # [ 148.705702] niks3-server[1004]: 2026/09/20 09:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5914worker1 # [ 148.752303] nix-grpc-daemon[1751]: time=2026-09-20T09:35:24.370Z level=INFO msg="Upload complete. (475ms)"5915lb # [ 148.710874] niks3-server[1004]: 2026/09/20 09:35:25 INFO Completed upload id=505916lb # [ 148.713952] niks3-server[1004]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5917worker1 # [ 148.760156] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=5125918worker1 # [ 148.771847] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765919worker1 # [ 148.778292] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5920worker1 # [ 148.783219] nix-daemon[836]: reaped child process 1834, status = succeeded5921worker1 # [ 148.796213] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5922worker1 # [ 149.618593] nix-daemon[836]: reaped child process 1848, status = succeeded5923worker1 # [ 149.782478] nix-daemon[836]: accepted connection from pid 1863, user nix-grpc-daemon (trusted)5924worker1 # [ 149.800520] nix-daemon[836]: reaped child process 1866, status = succeeded5925lb # [ 149.773904] niks3-server[1004]: 2026/09/20 09:35:26 INFO Received uploads request method=POST path=/api/pending_closures5926worker1 # [ 149.826162] nix-grpc-daemon[1751]: time=2026-09-20T09:35:25.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5927worker1 # [ 149.827616] nix-grpc-daemon[1751]: time=2026-09-20T09:35:25.442Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5928lb # [ 149.810641] niks3-server[1004]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5929lb # [ 149.821888] niks3-server[1004]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5930worker1 # [ 149.875262] nix-grpc-daemon[1751]: time=2026-09-20T09:35:25.493Z level=INFO msg="Uploading 1 narinfos"5931lb # [ 149.835954] niks3-server[1004]: 2026/09/20 09:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5932lb # [ 149.839372] niks3-server[1004]: 2026/09/20 09:35:26 INFO Signed narinfos id=51 count=15933lb # [ 149.848667] niks3-server[1004]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5934lb # [ 149.856828] niks3-server[1004]: 2026/09/20 09:35:26 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5935lb # [ 149.862002] niks3-server[1004]: 2026/09/20 09:35:26 INFO Completed upload id=515936lb # [ 149.864922] niks3-server[1004]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5937worker1 # [ 149.902105] nix-grpc-daemon[1751]: time=2026-09-20T09:35:25.520Z level=INFO msg="Upload complete. (296ms)"5938worker1 # [ 149.918336] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=15939worker1 # [ 149.936825] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:25Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15940worker1 # [ 149.941839] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765941worker1 # [ 149.946212] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:25Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05942worker1 # [ 149.950674] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456765943worker1 # [ 149.953851] nix-daemon[836]: reaped child process 1846, status = succeeded5944worker1 # [ 149.956581] nix-grpc-daemon[1751]: time=2026-09-20T09:35:25.574Z level=INFO msg="All 1 paths already cached"5945worker1 # [ 149.960361] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456765946worker1 # [ 149.972094] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5947worker1 # [ 150.154094] nix-daemon[836]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)5948worker1 # [ 150.171540] nix-daemon[836]: reaped child process 1875, status = succeeded5949worker1 # [ 150.348078] nix-daemon[836]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)5950worker1 # [ 150.364297] nix-daemon[836]: reaped child process 1880, status = succeeded5951lb # [ 150.336816] niks3-server[1004]: 2026/09/20 09:35:26 INFO Received uploads request method=POST path=/api/pending_closures5952worker1 # [ 150.387141] nix-grpc-daemon[1751]: time=2026-09-20T09:35:26.005Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5953worker1 # [ 150.388742] nix-grpc-daemon[1751]: time=2026-09-20T09:35:26.005Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"5954lb # [ 150.362708] niks3-server[1004]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst5955lb # [ 150.377179] niks3-server[1004]: 2026/09/20 09:35:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5956worker1 # [ 150.424237] nix-grpc-daemon[1751]: time=2026-09-20T09:35:26.042Z level=INFO msg="Uploading 1 narinfos"5957lb # [ 150.389307] niks3-server[1004]: 2026/09/20 09:35:27 INFO Signed narinfos id=52 count=15958lb # [ 150.395018] niks3-server[1004]: 2026/09/20 09:35:27 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls5959lb # [ 150.401803] niks3-server[1004]: 2026/09/20 09:35:27 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5960worker1 # [ 150.448616] nix-grpc-daemon[1751]: time=2026-09-20T09:35:26.066Z level=INFO msg="Upload complete. (465ms)"5961lb # [ 150.408004] niks3-server[1004]: 2026/09/20 09:35:27 INFO Completed upload id=525962lb # [ 150.411480] niks3-server[1004]: 2026/09/20 09:35:27 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo5963worker1 # [ 150.456132] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=6245964worker1 # [ 150.468156] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765965worker1 # [ 150.473680] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5966worker1 # [ 150.478217] nix-daemon[836]: reaped child process 1870, status = succeeded5967worker1 # [ 150.490423] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5968worker1 # [ 151.518267] nix-daemon[836]: reaped child process 1884, status = succeeded5969worker1 # [ 151.677483] nix-daemon[836]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)5970worker1 # [ 151.695868] nix-daemon[836]: reaped child process 1902, status = succeeded5971lb # [ 151.671037] niks3-server[1004]: 2026/09/20 09:35:28 INFO Received uploads request method=POST path=/api/pending_closures5972worker1 # [ 151.723893] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.339Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5973worker1 # [ 151.725595] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.339Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"5974lb # [ 151.704928] niks3-server[1004]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv5975lb # [ 151.716367] niks3-server[1004]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst5976worker1 # [ 151.771330] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.389Z level=INFO msg="Uploading 1 narinfos"5977lb # [ 151.733145] niks3-server[1004]: 2026/09/20 09:35:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign5978lb # [ 151.741855] niks3-server[1004]: 2026/09/20 09:35:28 INFO Signed narinfos id=53 count=15979lb # [ 151.750300] niks3-server[1004]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls5980lb # [ 151.754662] niks3-server[1004]: 2026/09/20 09:35:28 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete5981lb # [ 151.760186] niks3-server[1004]: 2026/09/20 09:35:28 INFO Completed upload id=535982worker1 # [ 151.801118] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.419Z level=INFO msg="Upload complete. (300ms)"5983lb # [ 151.762834] niks3-server[1004]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo5984worker1 # [ 151.820998] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=15985worker1 # [ 151.836965] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=15986worker1 # [ 151.844149] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456765987worker1 # [ 151.847236] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=05988worker1 # [ 151.851719] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456765989worker1 # [ 151.856449] nix-daemon[836]: reaped child process 1882, status = succeeded5990worker1 # [ 151.859574] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.477Z level=INFO msg="All 1 paths already cached"5991worker1 # [ 151.866637] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456765992worker1 # [ 151.878669] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)5993worker1 # [ 152.070106] nix-daemon[836]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)5994worker1 # [ 152.088330] nix-daemon[836]: reaped child process 1911, status = succeeded5995worker1 # [ 152.264089] nix-daemon[836]: accepted connection from pid 1913, user nix-grpc-daemon (trusted)5996worker1 # [ 152.280798] nix-daemon[836]: reaped child process 1916, status = succeeded5997lb # [ 152.251925] niks3-server[1004]: 2026/09/20 09:35:28 INFO Received uploads request method=POST path=/api/pending_closures5998worker1 # [ 152.302812] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.920Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5999worker1 # [ 152.304414] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.920Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6000lb # [ 152.277856] niks3-server[1004]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6001lb # [ 152.290916] niks3-server[1004]: 2026/09/20 09:35:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6002worker1 # [ 152.339425] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.957Z level=INFO msg="Uploading 1 narinfos"6003lb # [ 152.304214] niks3-server[1004]: 2026/09/20 09:35:28 INFO Signed narinfos id=54 count=16004lb # [ 152.306910] niks3-server[1004]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6005worker1 # [ 152.359168] nix-grpc-daemon[1751]: time=2026-09-20T09:35:27.977Z level=INFO msg="Upload complete. (467ms)"6006lb # [ 152.318303] niks3-server[1004]: 2026/09/20 09:35:28 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6007lb # [ 152.322752] niks3-server[1004]: 2026/09/20 09:35:28 INFO Completed upload id=546008lb # [ 152.326307] niks3-server[1004]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6009worker1 # [ 152.367857] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=6246010worker1 # [ 152.378206] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766011worker1 # [ 152.383601] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6012worker1 # [ 152.387593] nix-daemon[836]: reaped child process 1906, status = succeeded6013worker1 # [ 152.397337] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6014worker1 # [ 153.825286] nix-daemon[836]: reaped child process 1920, status = succeeded6015worker1 # [ 153.998333] nix-daemon[836]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6016worker1 # [ 154.016622] nix-daemon[836]: reaped child process 1938, status = succeeded6017lb # [ 153.990322] niks3-server[1004]: 2026/09/20 09:35:30 INFO Received uploads request method=POST path=/api/pending_closures6018worker1 # [ 154.044294] nix-grpc-daemon[1751]: time=2026-09-20T09:35:29.659Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6019worker1 # [ 154.045807] nix-grpc-daemon[1751]: time=2026-09-20T09:35:29.659Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6020lb # [ 154.024655] niks3-server[1004]: 2026/09/20 09:35:30 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6021worker1 # [ 154.081130] nix-grpc-daemon[1751]: time=2026-09-20T09:35:29.698Z level=INFO msg="Uploading 1 narinfos"6022lb # [ 154.039206] niks3-server[1004]: 2026/09/20 09:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6023lb # [ 154.052629] niks3-server[1004]: 2026/09/20 09:35:30 INFO Signed narinfos id=55 count=16024lb # [ 154.055558] niks3-server[1004]: 2026/09/20 09:35:30 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6025lb # [ 154.066452] niks3-server[1004]: 2026/09/20 09:35:30 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6026lb # [ 154.080093] niks3-server[1004]: 2026/09/20 09:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6027lb # [ 154.083709] niks3-server[1004]: 2026/09/20 09:35:30 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6028worker1 # [ 154.129396] nix-grpc-daemon[1751]: time=2026-09-20T09:35:29.747Z level=INFO msg="Upload complete. (316ms)"6029lb # [ 154.090811] niks3-server[1004]: 2026/09/20 09:35:30 INFO Completed upload id=556030worker1 # [ 154.145387] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16031worker1 # [ 154.158835] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:29Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16032worker1 # [ 154.168279] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766033worker1 # [ 154.171795] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:29Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06034worker1 # [ 154.177879] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456766035worker1 # [ 154.182135] nix-grpc-daemon[1751]: time=2026-09-20T09:35:29.799Z level=INFO msg="All 1 paths already cached"6036worker1 # [ 154.184812] nix-daemon[836]: reaped child process 1918, status = succeeded6037worker1 # [ 154.187395] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456766038worker1 # [ 154.197889] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6039worker1 # [ 154.382778] nix-daemon[836]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6040worker1 # [ 154.403546] nix-daemon[836]: reaped child process 1947, status = succeeded6041worker1 # [ 154.572823] nix-daemon[836]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6042worker1 # [ 154.592166] nix-daemon[836]: reaped child process 1952, status = succeeded6043lb # [ 154.562020] niks3-server[1004]: 2026/09/20 09:35:31 INFO Received uploads request method=POST path=/api/pending_closures6044worker1 # [ 154.612753] nix-grpc-daemon[1751]: time=2026-09-20T09:35:30.230Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6045worker1 # [ 154.614241] nix-grpc-daemon[1751]: time=2026-09-20T09:35:30.230Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6046lb # [ 154.589000] niks3-server[1004]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6047lb # [ 154.603150] niks3-server[1004]: 2026/09/20 09:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6048worker1 # [ 154.650785] nix-grpc-daemon[1751]: time=2026-09-20T09:35:30.268Z level=INFO msg="Uploading 1 narinfos"6049lb # [ 154.616368] niks3-server[1004]: 2026/09/20 09:35:31 INFO Signed narinfos id=56 count=16050lb # [ 154.622118] niks3-server[1004]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6051worker1 # [ 154.671157] nix-grpc-daemon[1751]: time=2026-09-20T09:35:30.289Z level=INFO msg="Upload complete. (462ms)"6052lb # [ 154.630625] niks3-server[1004]: 2026/09/20 09:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6053lb # [ 154.635301] niks3-server[1004]: 2026/09/20 09:35:31 INFO Completed upload id=566054lb # [ 154.638320] niks3-server[1004]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6055worker1 # [ 154.680189] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=6246056worker1 # [ 154.683854] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766057worker1 # [ 154.695641] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6058worker1 # [ 154.699721] nix-daemon[836]: reaped child process 1942, status = succeeded6059worker1 # [ 154.712700] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6060worker1 # [ 155.745063] nix-daemon[836]: reaped child process 1956, status = succeeded6061worker1 # [ 155.912335] nix-daemon[836]: accepted connection from pid 1971, user nix-grpc-daemon (trusted)6062worker1 # [ 155.929039] nix-daemon[836]: reaped child process 1974, status = succeeded6063lb # [ 155.902386] niks3-server[1004]: 2026/09/20 09:35:32 INFO Received uploads request method=POST path=/api/pending_closures6064worker1 # [ 155.955556] nix-grpc-daemon[1751]: time=2026-09-20T09:35:31.572Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6065worker1 # [ 155.957363] nix-grpc-daemon[1751]: time=2026-09-20T09:35:31.572Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6066lb # [ 155.940096] niks3-server[1004]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6067lb # [ 155.950640] niks3-server[1004]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6068lb # [ 155.960608] niks3-server[1004]: 2026/09/20 09:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6069worker1 # [ 156.008145] nix-grpc-daemon[1751]: time=2026-09-20T09:35:31.626Z level=INFO msg="Uploading 1 narinfos"6070lb # [ 155.966714] niks3-server[1004]: 2026/09/20 09:35:32 INFO Signed narinfos id=57 count=16071lb # [ 155.973109] niks3-server[1004]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6072lb # [ 155.984221] niks3-server[1004]: 2026/09/20 09:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6073lb # [ 155.991286] niks3-server[1004]: 2026/09/20 09:35:32 INFO Completed upload id=576074worker1 # [ 156.032743] nix-grpc-daemon[1751]: time=2026-09-20T09:35:31.650Z level=INFO msg="Upload complete. (310ms)"6075lb # [ 155.994643] niks3-server[1004]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6076worker1 # [ 156.049893] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16077worker1 # [ 156.064818] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:31Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16078worker1 # [ 156.071971] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766079worker1 # [ 156.075587] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:31Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06080worker1 # [ 156.079481] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456766081worker1 # [ 156.083888] nix-daemon[836]: reaped child process 1954, status = succeeded6082worker1 # [ 156.087739] nix-grpc-daemon[1751]: time=2026-09-20T09:35:31.705Z level=INFO msg="All 2 paths already cached"6083worker1 # [ 156.094617] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456766084worker1 # [ 156.106084] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6085worker1 # [ 156.291385] nix-daemon[836]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6086worker1 # [ 156.310480] nix-daemon[836]: reaped child process 1983, status = succeeded6087worker1 # [ 156.494224] nix-daemon[836]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6088worker1 # [ 156.510495] nix-daemon[836]: reaped child process 1988, status = succeeded6089lb # [ 156.480757] niks3-server[1004]: 2026/09/20 09:35:33 INFO Received uploads request method=POST path=/api/pending_closures6090worker1 # [ 156.530698] nix-grpc-daemon[1751]: time=2026-09-20T09:35:32.148Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6091worker1 # [ 156.532269] nix-grpc-daemon[1751]: time=2026-09-20T09:35:32.148Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6092lb # [ 156.506926] niks3-server[1004]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6093lb # [ 156.520862] niks3-server[1004]: 2026/09/20 09:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6094worker1 # [ 156.569182] nix-grpc-daemon[1751]: time=2026-09-20T09:35:32.187Z level=INFO msg="Uploading 1 narinfos"6095lb # [ 156.527920] niks3-server[1004]: 2026/09/20 09:35:33 INFO Signed narinfos id=58 count=16096lb # [ 156.535699] niks3-server[1004]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6097lb # [ 156.544855] niks3-server[1004]: 2026/09/20 09:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6098worker1 # [ 156.591056] nix-grpc-daemon[1751]: time=2026-09-20T09:35:32.209Z level=INFO msg="Upload complete. (474ms)"6099lb # [ 156.550183] niks3-server[1004]: 2026/09/20 09:35:33 INFO Completed upload id=586100lb # [ 156.552769] niks3-server[1004]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6101worker1 # [ 156.598413] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=7526102worker1 # [ 156.609659] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766103worker1 # [ 156.615997] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6104worker1 # [ 156.620681] nix-daemon[836]: reaped child process 1978, status = succeeded6105worker1 # [ 156.632978] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6106worker1 # [ 157.868001] nix-daemon[836]: reaped child process 1992, status = succeeded6107worker1 # [ 158.033570] nix-daemon[836]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)6108worker1 # [ 158.052299] nix-daemon[836]: reaped child process 2010, status = succeeded6109lb # [ 158.026097] niks3-server[1004]: 2026/09/20 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures6110worker1 # [ 158.080405] nix-grpc-daemon[1751]: time=2026-09-20T09:35:33.695Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6111worker1 # [ 158.081996] nix-grpc-daemon[1751]: time=2026-09-20T09:35:33.695Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6112lb # [ 158.064151] niks3-server[1004]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6113lb # [ 158.076748] niks3-server[1004]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6114lb # [ 158.084558] niks3-server[1004]: 2026/09/20 09:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6115worker1 # [ 158.130897] nix-grpc-daemon[1751]: time=2026-09-20T09:35:33.748Z level=INFO msg="Uploading 1 narinfos"6116lb # [ 158.094304] niks3-server[1004]: 2026/09/20 09:35:34 INFO Signed narinfos id=59 count=16117lb # [ 158.100936] niks3-server[1004]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6118lb # [ 158.108337] niks3-server[1004]: 2026/09/20 09:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6119lb # [ 158.116671] niks3-server[1004]: 2026/09/20 09:35:34 INFO Completed upload id=596120worker1 # [ 158.157224] nix-grpc-daemon[1751]: time=2026-09-20T09:35:33.775Z level=INFO msg="Upload complete. (313ms)"6121lb # [ 158.119394] niks3-server[1004]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6122worker1 # [ 158.173092] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16123worker1 # [ 158.186417] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:33Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16124worker1 # [ 158.191133] nix-daemon[836]: reaped child process 1990, status = succeeded6125worker1 # [ 158.213496] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766126worker1 # [ 158.228241] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456766127worker1 # [ 158.241852] nix-grpc-daemon[1751]: time=2026-09-20T09:35:33.860Z level=INFO msg="All 2 paths already cached"6128worker1 # [ 158.251075] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456766129worker1 # [ 158.264327] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6130worker1 # [ 158.464912] nix-daemon[836]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6131worker1 # [ 158.482100] nix-daemon[836]: reaped child process 2017, status = succeeded6132worker1 # [ 158.655322] nix-daemon[836]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)6133worker1 # [ 158.671695] nix-daemon[836]: reaped child process 2022, status = succeeded6134lb # [ 158.645893] niks3-server[1004]: 2026/09/20 09:35:35 INFO Received uploads request method=POST path=/api/pending_closures6135worker1 # [ 158.697707] nix-grpc-daemon[1751]: time=2026-09-20T09:35:34.315Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6136worker1 # [ 158.699217] nix-grpc-daemon[1751]: time=2026-09-20T09:35:34.315Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6137lb # [ 158.675857] niks3-server[1004]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6138lb # [ 158.687044] niks3-server[1004]: 2026/09/20 09:35:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6139worker1 # [ 158.735230] nix-grpc-daemon[1751]: time=2026-09-20T09:35:34.353Z level=INFO msg="Uploading 1 narinfos"6140lb # [ 158.699138] niks3-server[1004]: 2026/09/20 09:35:35 INFO Signed narinfos id=60 count=16141lb # [ 158.705154] niks3-server[1004]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6142lb # [ 158.714663] niks3-server[1004]: 2026/09/20 09:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6143worker1 # [ 158.760217] nix-grpc-daemon[1751]: time=2026-09-20T09:35:34.378Z level=INFO msg="Upload complete. (479ms)"6144lb # [ 158.719259] niks3-server[1004]: 2026/09/20 09:35:35 INFO Completed upload id=606145lb # [ 158.722038] niks3-server[1004]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6146worker1 # [ 158.768127] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=7526147worker1 # [ 158.779408] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766148worker1 # [ 158.785660] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6149worker1 # [ 158.790768] nix-daemon[836]: reaped child process 2012, status = succeeded6150worker1 # [ 158.803989] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6151worker1 # [ 160.224620] nix-daemon[836]: reaped child process 2026, status = succeeded6152worker1 # [ 160.381158] nix-daemon[836]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6153worker1 # [ 160.399689] nix-daemon[836]: reaped child process 2044, status = succeeded6154lb # [ 160.372782] niks3-server[1004]: 2026/09/20 09:35:36 INFO Received uploads request method=POST path=/api/pending_closures6155worker1 # [ 160.423915] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.040Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6156worker1 # [ 160.425596] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.041Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6157lb # [ 160.409232] niks3-server[1004]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6158lb # [ 160.425230] niks3-server[1004]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6159lb # [ 160.433021] niks3-server[1004]: 2026/09/20 09:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6160worker1 # [ 160.478999] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.096Z level=INFO msg="Uploading 1 narinfos"6161lb # [ 160.441709] niks3-server[1004]: 2026/09/20 09:35:37 INFO Signed narinfos id=61 count=16162lb # [ 160.443725] niks3-server[1004]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6163lb # [ 160.451419] niks3-server[1004]: 2026/09/20 09:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6164lb # [ 160.455811] niks3-server[1004]: 2026/09/20 09:35:37 INFO Completed upload id=616165worker1 # [ 160.497250] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.115Z level=INFO msg="Upload complete. (283ms)"6166lb # [ 160.458101] niks3-server[1004]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6167worker1 # [ 160.508975] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16168worker1 # [ 160.520240] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:36Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16169worker1 # [ 160.526478] nix-daemon[836]: reaped child process 2024, status = succeeded6170worker1 # [ 160.549856] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766171worker1 # [ 160.563844] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456766172worker1 # [ 160.577752] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.195Z level=INFO msg="All 2 paths already cached"6173worker1 # [ 160.587090] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:456766174worker1 # [ 160.601813] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6175worker1 # [ 160.799308] nix-daemon[836]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)6176worker1 # [ 160.816363] nix-daemon[836]: reaped child process 2051, status = succeeded6177worker1 # [ 160.984834] nix-daemon[836]: accepted connection from pid 2053, user nix-grpc-daemon (trusted)6178worker1 # [ 161.000555] nix-daemon[836]: reaped child process 2056, status = succeeded6179lb # [ 160.972572] niks3-server[1004]: 2026/09/20 09:35:37 INFO Received uploads request method=POST path=/api/pending_closures6180worker1 # [ 161.022534] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.640Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6181worker1 # [ 161.024000] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.640Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6182lb # [ 160.998361] niks3-server[1004]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6183lb # [ 161.012302] niks3-server[1004]: 2026/09/20 09:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6184worker1 # [ 161.059116] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.677Z level=INFO msg="Uploading 1 narinfos"6185lb # [ 161.022336] niks3-server[1004]: 2026/09/20 09:35:37 INFO Signed narinfos id=62 count=16186lb # [ 161.028781] niks3-server[1004]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6187lb # [ 161.040288] niks3-server[1004]: 2026/09/20 09:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6188worker1 # [ 161.085918] nix-grpc-daemon[1751]: time=2026-09-20T09:35:36.704Z level=INFO msg="Upload complete. (468ms)"6189worker1 # [ 161.091161] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=0 paths=1 nar_bytes_in=7526190lb # [ 161.049778] niks3-server[1004]: 2026/09/20 09:35:37 INFO Completed upload id=626191lb # [ 161.054115] niks3-server[1004]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6192worker1 # [ 161.102957] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:456766193worker1 # [ 161.109766] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6194worker1 # [ 161.114376] nix-daemon[836]: reaped child process 2046, status = succeeded6195worker1 # [ 161.126888] nix-daemon[836]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6196worker1 # [ 162.769608] nix-daemon[836]: reaped child process 2060, status = succeeded6197worker1 # [ 162.951783] nix-daemon[836]: accepted connection from pid 2075, user nix-grpc-daemon (trusted)6198worker1 # [ 162.968205] nix-daemon[836]: reaped child process 2078, status = succeeded6199lb # [ 162.944903] niks3-server[1004]: 2026/09/20 09:35:39 INFO Received uploads request method=POST path=/api/pending_closures6200worker1 # [ 162.996577] nix-grpc-daemon[1751]: time=2026-09-20T09:35:38.612Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6201worker1 # [ 162.998740] nix-grpc-daemon[1751]: time=2026-09-20T09:35:38.612Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6202lb # [ 162.984007] niks3-server[1004]: 2026/09/20 09:35:39 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6203lb # [ 162.998038] niks3-server[1004]: 2026/09/20 09:35:39 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6204worker1 # [ 163.047620] nix-grpc-daemon[1751]: time=2026-09-20T09:35:38.665Z level=INFO msg="Uploading 1 narinfos"6205lb # [ 163.008271] niks3-server[1004]: 2026/09/20 09:35:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6206lb # [ 163.017384] niks3-server[1004]: 2026/09/20 09:35:39 INFO Signed narinfos id=63 count=16207lb # [ 163.027146] niks3-server[1004]: 2026/09/20 09:35:39 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6208worker1 # [ 163.073953] nix-grpc-daemon[1751]: time=2026-09-20T09:35:38.692Z level=INFO msg="Upload complete. (335ms)"6209lb # [ 163.032942] niks3-server[1004]: 2026/09/20 09:35:39 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6210lb # [ 163.037250] niks3-server[1004]: 2026/09/20 09:35:39 INFO Completed upload id=636211lb # [ 163.041184] niks3-server[1004]: 2026/09/20 09:35:39 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6212worker1 # [ 163.087669] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45676 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16213worker1 # [ 163.106186] nix-daemon[836]: reaped child process 2058, status = succeeded6214client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.38 seconds)6215worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6216worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.08 seconds)6217client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16218worker1 # [ 163.487230] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:456766219worker1 # [ 163.495286] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456766220worker1 # [ 163.512994] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:39Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06221client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.30 seconds)6222(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 21.77 seconds)6223subtest: low disk drains a worker and builds go to the other6224worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6225worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.12 seconds)6226worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6227worker1 # [ 165.441707] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:41Z level=info event=unhealthy reason=min_free6228worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6229client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26230worker2 # [ 165.979011] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500286231client # don't know how to build these paths:6232client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6233worker1 # [ 166.043186] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:456766234worker1 # [ 166.067844] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:41Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06235worker1 # [ 166.073897] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:41Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16236worker2 # [ 166.062771] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500286237worker2 # [ 166.079600] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500286238client # copying 1 paths to 'lb:50051'...6239client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6240worker2 # [ 166.096081] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500286241worker2 # [ 166.111992] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6242worker2 # [ 166.335367] nix-daemon[832]: accepted connection from pid 1610, user nix-grpc-daemon (trusted)6243worker2 # [ 166.355904] nix-daemon[832]: reaped child process 1613, status = succeeded6244worker2 # [ 166.534731] nix-daemon[832]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)6245worker2 # [ 166.550781] nix-daemon[832]: reaped child process 1618, status = succeeded6246lb # [ 166.578135] niks3-server[1004]: 2026/09/20 09:35:43 INFO Received uploads request method=POST path=/api/pending_closures6247worker2 # [ 166.579592] nix-grpc-daemon[1425]: time=2026-09-20T09:35:43.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6248worker2 # [ 166.581209] nix-grpc-daemon[1425]: time=2026-09-20T09:35:43.206Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6249lb # [ 166.607347] niks3-server[1004]: 2026/09/20 09:35:43 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6250lb # [ 166.622202] niks3-server[1004]: 2026/09/20 09:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6251worker2 # [ 166.618158] nix-grpc-daemon[1425]: time=2026-09-20T09:35:43.245Z level=INFO msg="Uploading 1 narinfos"6252lb # [ 166.634812] niks3-server[1004]: 2026/09/20 09:35:43 INFO Signed narinfos id=64 count=16253lb # [ 166.642117] niks3-server[1004]: 2026/09/20 09:35:43 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6254worker2 # [ 166.642458] nix-grpc-daemon[1425]: time=2026-09-20T09:35:43.270Z level=INFO msg="Upload complete. (509ms)"6255lb # [ 166.652844] niks3-server[1004]: 2026/09/20 09:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6256lb # [ 166.658836] niks3-server[1004]: 2026/09/20 09:35:43 INFO Completed upload id=646257worker2 # [ 166.652246] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=0 paths=1 nar_bytes_in=5126258lb # [ 166.663018] niks3-server[1004]: 2026/09/20 09:35:43 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6259worker2 # [ 166.663330] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500286260worker2 # [ 166.672246] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6261worker2 # [ 166.675363] nix-daemon[832]: reaped child process 1608, status = succeeded6262client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6263worker2 # [ 166.688165] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6264client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6265client # farm-a> LOG-farm-a-drain6266worker2 # [ 168.334968] nix-daemon[832]: reaped child process 1622, status = succeeded6267worker2 # [ 168.513308] nix-daemon[832]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)6268worker2 # [ 168.530958] nix-daemon[832]: reaped child process 1640, status = succeeded6269lb # [ 168.554673] niks3-server[1004]: 2026/09/20 09:35:45 INFO Received uploads request method=POST path=/api/pending_closures6270worker2 # [ 168.555950] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6271worker2 # [ 168.557575] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.181Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6272lb # [ 168.585479] niks3-server[1004]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6273lb # [ 168.598222] niks3-server[1004]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6274lb # [ 168.612232] niks3-server[1004]: 2026/09/20 09:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6275worker2 # [ 168.608185] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.235Z level=INFO msg="Uploading 1 narinfos"6276lb # [ 168.622933] niks3-server[1004]: 2026/09/20 09:35:45 INFO Signed narinfos id=65 count=16277lb # [ 168.628655] niks3-server[1004]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6278lb # [ 168.638104] niks3-server[1004]: 2026/09/20 09:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6279lb # [ 168.643237] niks3-server[1004]: 2026/09/20 09:35:45 INFO Completed upload id=656280worker2 # [ 168.632566] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.260Z level=INFO msg="Upload complete. (334ms)"6281lb # [ 168.646529] niks3-server[1004]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6282worker2 # [ 168.644979] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=16283worker1 # [ 168.703580] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:44Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06284worker1 # [ 168.709601] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:44Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16285worker2 # [ 168.662382] nix-daemon[832]: reaped child process 1620, status = succeeded6286worker2 # [ 168.698768] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500286287worker2 # [ 168.714800] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500286288worker2 # [ 168.728470] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.356Z level=INFO msg="All 1 paths already cached"6289client # copying 1 paths to 'lb:50051'...6290client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6291worker2 # [ 168.738228] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500286292worker2 # [ 168.751900] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6293worker2 # [ 168.957407] nix-daemon[832]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6294worker2 # [ 168.976455] nix-daemon[832]: reaped child process 1647, status = succeeded6295worker2 # [ 169.155620] nix-daemon[832]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6296worker2 # [ 169.174865] nix-daemon[832]: reaped child process 1652, status = succeeded6297lb # [ 169.196424] niks3-server[1004]: 2026/09/20 09:35:45 INFO Received uploads request method=POST path=/api/pending_closures6298worker2 # [ 169.196188] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.824Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6299worker2 # [ 169.198158] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.825Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6300lb # [ 169.227140] niks3-server[1004]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6301worker2 # [ 169.227917] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.855Z level=INFO msg="Uploading 1 narinfos"6302lb # [ 169.241427] niks3-server[1004]: 2026/09/20 09:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6303lb # [ 169.251327] niks3-server[1004]: 2026/09/20 09:35:45 INFO Signed narinfos id=66 count=16304lb # [ 169.256874] niks3-server[1004]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6305lb # [ 169.265646] niks3-server[1004]: 2026/09/20 09:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6306worker2 # [ 169.259702] nix-grpc-daemon[1425]: time=2026-09-20T09:35:45.887Z level=INFO msg="Upload complete. (491ms)"6307lb # [ 169.272807] niks3-server[1004]: 2026/09/20 09:35:45 INFO Completed upload id=666308lb # [ 169.275615] niks3-server[1004]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6309worker2 # [ 169.267070] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=0 paths=1 nar_bytes_in=6326310worker2 # [ 169.275267] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500286311worker2 # [ 169.285620] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6312worker2 # [ 169.290216] nix-daemon[832]: reaped child process 1642, status = succeeded6313client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6314client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6315worker2 # [ 169.303101] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6316lb # [ 169.403247] envoy[902]: conn peer=[2001:db8:1::1]:60658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27139 tx=10793 ms=210576317lb # [ 169.409062] envoy[902]: conn peer=[2001:db8:1::1]:33278 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=880 tx=482 ms=476318client # farm-b> LOG-farm-b-drain6319worker2 # [ 170.529188] nix-daemon[832]: reaped child process 1656, status = succeeded6320worker2 # [ 170.692751] nix-daemon[832]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6321worker2 # [ 170.711576] nix-daemon[832]: reaped child process 1674, status = succeeded6322lb # [ 170.733824] niks3-server[1004]: 2026/09/20 09:35:47 INFO Received uploads request method=POST path=/api/pending_closures6323worker2 # [ 170.736399] nix-grpc-daemon[1425]: time=2026-09-20T09:35:47.360Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6324worker2 # [ 170.737882] nix-grpc-daemon[1425]: time=2026-09-20T09:35:47.360Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6325lb # [ 170.765027] niks3-server[1004]: 2026/09/20 09:35:47 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6326lb # [ 170.775860] niks3-server[1004]: 2026/09/20 09:35:47 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6327lb # [ 170.791699] niks3-server[1004]: 2026/09/20 09:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6328worker2 # [ 170.786847] nix-grpc-daemon[1425]: time=2026-09-20T09:35:47.414Z level=INFO msg="Uploading 1 narinfos"6329lb # [ 170.801086] niks3-server[1004]: 2026/09/20 09:35:47 INFO Signed narinfos id=67 count=16330lb # [ 170.803549] niks3-server[1004]: 2026/09/20 09:35:47 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6331lb # [ 170.813610] niks3-server[1004]: 2026/09/20 09:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6332lb # [ 170.820726] niks3-server[1004]: 2026/09/20 09:35:47 INFO Completed upload id=676333worker2 # [ 170.810775] nix-grpc-daemon[1425]: time=2026-09-20T09:35:47.438Z level=INFO msg="Upload complete. (298ms)"6334lb # [ 170.823670] niks3-server[1004]: 2026/09/20 09:35:47 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6335worker1 # [ 170.880333] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:46Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06336worker2 # [ 170.820998] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=16337worker2 # [ 170.837637] nix-daemon[832]: reaped child process 1654, status = succeeded6338worker1 # [ 170.885787] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:46Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16339worker2 # [ 170.873869] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500286340worker2 # [ 170.889405] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500286341worker2 # [ 170.903390] nix-grpc-daemon[1425]: time=2026-09-20T09:35:47.531Z level=INFO msg="All 2 paths already cached"6342client # copying 1 paths to 'lb:50051'...6343client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6344worker2 # [ 170.913622] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500286345worker2 # [ 170.927334] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6346worker2 # [ 171.134549] nix-daemon[832]: accepted connection from pid 1678, user nix-grpc-daemon (trusted)6347worker2 # [ 171.155731] nix-daemon[832]: reaped child process 1681, status = succeeded6348worker2 # [ 171.331208] nix-daemon[832]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6349worker2 # [ 171.351008] nix-daemon[832]: reaped child process 1686, status = succeeded6350lb # [ 171.372468] niks3-server[1004]: 2026/09/20 09:35:47 INFO Received uploads request method=POST path=/api/pending_closures6351worker2 # [ 171.371416] nix-grpc-daemon[1425]: time=2026-09-20T09:35:47.999Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6352worker2 # [ 171.373089] nix-grpc-daemon[1425]: time=2026-09-20T09:35:47.999Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6353lb # [ 171.400395] niks3-server[1004]: 2026/09/20 09:35:48 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6354lb # [ 171.412775] niks3-server[1004]: 2026/09/20 09:35:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6355worker2 # [ 171.408612] nix-grpc-daemon[1425]: time=2026-09-20T09:35:48.036Z level=INFO msg="Uploading 1 narinfos"6356lb # [ 171.424465] niks3-server[1004]: 2026/09/20 09:35:48 INFO Signed narinfos id=68 count=16357lb # [ 171.431378] niks3-server[1004]: 2026/09/20 09:35:48 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6358lb # [ 171.442109] niks3-server[1004]: 2026/09/20 09:35:48 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6359worker2 # [ 171.437928] nix-grpc-daemon[1425]: time=2026-09-20T09:35:48.065Z level=INFO msg="Upload complete. (493ms)"6360lb # [ 171.447907] niks3-server[1004]: 2026/09/20 09:35:48 INFO Completed upload id=686361lb # [ 171.452190] niks3-server[1004]: 2026/09/20 09:35:48 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6362worker2 # [ 171.444227] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=0 paths=1 nar_bytes_in=7606363worker2 # [ 171.456193] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500286364worker2 # [ 171.462457] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6365worker2 # [ 171.467222] nix-daemon[832]: reaped child process 1676, status = succeeded6366client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6367worker2 # [ 171.478652] nix-daemon[832]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)6368client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6369client # farm-top> LOG-farm-top-drain6370worker2 # [ 172.306846] nix-daemon[832]: reaped child process 1690, status = succeeded6371worker2 # [ 172.462935] nix-daemon[832]: accepted connection from pid 1705, user nix-grpc-daemon (trusted)6372worker2 # [ 172.482404] nix-daemon[832]: reaped child process 1708, status = succeeded6373lb # [ 172.504897] niks3-server[1004]: 2026/09/20 09:35:49 INFO Received uploads request method=POST path=/api/pending_closures6374worker2 # [ 172.505001] nix-grpc-daemon[1425]: time=2026-09-20T09:35:49.131Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6375worker2 # [ 172.506479] nix-grpc-daemon[1425]: time=2026-09-20T09:35:49.131Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6376lb # [ 172.534564] niks3-server[1004]: 2026/09/20 09:35:49 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6377lb # [ 172.547391] niks3-server[1004]: 2026/09/20 09:35:49 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6378lb # [ 172.560458] niks3-server[1004]: 2026/09/20 09:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6379worker2 # [ 172.556468] nix-grpc-daemon[1425]: time=2026-09-20T09:35:49.184Z level=INFO msg="Uploading 1 narinfos"6380lb # [ 172.570665] niks3-server[1004]: 2026/09/20 09:35:49 INFO Signed narinfos id=69 count=16381lb # [ 172.576257] niks3-server[1004]: 2026/09/20 09:35:49 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6382lb # [ 172.583786] niks3-server[1004]: 2026/09/20 09:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6383lb # [ 172.591073] niks3-server[1004]: 2026/09/20 09:35:49 INFO Completed upload id=696384worker2 # [ 172.581193] nix-grpc-daemon[1425]: time=2026-09-20T09:35:49.209Z level=INFO msg="Upload complete. (290ms)"6385lb # [ 172.593938] niks3-server[1004]: 2026/09/20 09:35:49 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6386worker2 # [ 172.590169] nix-grpc-daemon[1291]: ts=2026-09-20T09:35:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50028 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=16387worker2 # [ 172.606189] nix-daemon[832]: reaped child process 1688, status = succeeded6388client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.95 seconds)6389client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6390client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6391worker1: must succeed: rm /nix/.rw-store/fill6392worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6393worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6394worker1 # [ 173.451996] nix-grpc-daemon[1688]: ts=2026-09-20T09:35:49Z level=info event=healthy reason=min_free6395worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6396(finished: subtest: low disk drains a worker and builds go to the other, in 10.80 seconds)6397(finished: run the VM test script, in 175.18 seconds)6398test script finished in 175.48s6399cleanup6400kill QemuMachine (pid 51)6401client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6402kill QemuMachine (pid 49)6403lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6404kill QemuMachine (pid 50)6405worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6406kill QemuMachine (pid 52)6407worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6408(finished: cleanup, in 1.03 seconds)6409additionally exposed symbols:6410 client, lb, worker1, worker2,6411 vlan1,6412 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh