tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker1 # Disk image does not exist, creating the virtualisation disk image... worker2 # Disk image does not exist, creating the virtualisation disk image... worker2: QEMU running (pid 52) worker1: QEMU running (pid 51) worker1 # Formatting '/build/vm-state-worker1/tmp.xsLVQJybNj', fmt=raw size=1073741824 lb: QEMU running (pid 50) client # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 49) worker2 # Formatting '/build/vm-state-worker2/tmp.NMX0iu6wHb', fmt=raw size=1073741824 client # Formatting '/build/vm-state-client/tmp.yQbpMxBhEa', fmt=raw size=1073741824 lb # Disk image does not exist, creating the virtualisation disk image... worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 0.51 seconds) worker1 # Discarding device blocks: 0/262144 done worker2 # Discarding device blocks: 0/262144 done client: waiting for unit network-addresses-eth1.service worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client: waiting for the VM to finish booting worker1 # Filesystem UUID: 85753b96-c7ba-477e-964a-82432a2541e5 worker2 # Filesystem UUID: e2934ba0-53e2-4b9d-9be0-9be4a9393436 lb # Formatting '/build/vm-state-lb/tmp.8UihXkIwnz', fmt=raw size=1073741824 worker1 # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: client # mke2fs 1.47.4 (6-Mar-2025) worker2 # 32768, 98304, 163840, 229376 client # Discarding device blocks: 0/262144 done worker1 # 32768, 98304, 163840, 229376 lb # mke2fs 1.47.4 (6-Mar-2025) worker2 # client # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # lb # Discarding device blocks: 0/262144 done worker2 # Allocating group tables: 0/8 done client # Filesystem UUID: 029cd0ef-f3c7-49c5-af97-28279bde576d worker1 # Allocating group tables: 0/8 done lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Writing inode tables: 0/8 done client # Superblock backups stored on blocks: worker1 # Writing inode tables: 0/8 done lb # Filesystem UUID: 14220a4a-ec47-41b7-9745-bca8042d2bff worker2 # Creating journal (8192 blocks): done client # 32768, 98304, 163840, 229376 worker1 # Creating journal (8192 blocks): done lb # Superblock backups stored on blocks: worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # worker1 # Writing superblocks and filesystem accounting information: 0/8 done lb # 32768, 98304, 163840, 229376 worker2 # client # Allocating group tables: 0/8 done worker1 # lb # worker2 # Virtualisation disk image created. client # Writing inode tables: 0/8 done worker1 # Virtualisation disk image created. lb # Allocating group tables: 0/8 done worker2 # Starting virtiofs daemons... client # Creating journal (8192 blocks): done worker1 # Starting virtiofs daemons... lb # Writing inode tables: 0/8 done worker2 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # Writing superblocks and filesystem accounting information: 0/8 done worker1 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Creating journal (8192 blocks): done worker2 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # worker1 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Writing superblocks and filesystem accounting information: 0/8 done worker2 # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # Virtualisation disk image created. worker1 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # worker2 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # Starting virtiofs daemons... worker1 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Virtualisation disk image created. worker2 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Starting virtiofs daemons... worker2 # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:24:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:24:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:24:58Z INFO virtiofsd] Client connected, servicing requests client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] KASLR enabled client # [ 0.000000] random: crng init done client # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] efi: UEFI not found. client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty client # [ 0.000000] Normal empty client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node client # [ 0.000000] Early memory node ranges client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test/init regInfo=/nix/store/p2989fx14h8v8fmf8spd0dk73x9d2np7-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/p2989fx14h8v8fmf8spd0dk73x9d2np7-closure-info/registration", will be passed to user space. client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 client # [ 0.000000] Policy zone: DMA client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000033] arm-pv: using stolen time PV client # [ 0.000649] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.000815] Console: colour dummy device 80x25 client # [ 0.000822] printk: legacy console [tty0] enabled client # [ 0.001038] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.001044] pid_max: default: 32768 minimum: 301 client # [ 0.001131] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.001326] landlock: Up and running. client # [ 0.001329] Yama: becoming mindful. client # [ 0.001981] LSM support for eBPF active client # [ 0.002132] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.002165] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.003971] rcu: Hierarchical SRCU implementation. client # [ 0.003975] rcu: Max phase no-delay instances is 1000. client # [ 0.005249] fsl-mc MSI: its@8080000 domain created client # [ 0.005339] EFI services will not be available. client # [ 0.005447] smp: Bringing up secondary CPUs ... client # [ 0.005455] smp: Brought up 1 node, 1 CPU client # [ 0.005458] SMP: Total of 1 processors activated. client # [ 0.005463] CPU: All CPU(s) started at EL1 client # [ 0.005474] CPU features: detected: Branch Target Identification client # [ 0.005478] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.005482] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.005485] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.005489] CPU features: detected: Common not Private translations client # [ 0.005492] CPU features: detected: CRC32 instructions client # [ 0.005495] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.005499] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.005502] CPU features: detected: Data independent timing control (DIT) client # [ 0.005505] CPU features: detected: E0PD client # [ 0.005507] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.005510] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005514] CPU features: detected: Enhanced Virtualization Traps client # [ 0.005517] CPU features: detected: Fine Grained Traps worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] KASLR enabled worker1 # [ 0.000000] random: crng init done client # [ 0.005520] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] efi: UEFI not found. client # [ 0.005525] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.005528] CPU features: detected: LSE atomic instructions worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.005531] CPU features: detected: Privileged Access Never worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005533] CPU features: detected: PMUv3 worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.005536] CPU features: detected: RAS Extension Support worker1 # [ 0.000000] Zone ranges: client # [ 0.005539] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] DMA32 empty client # [ 0.005542] CPU features: detected: Random Number Generator worker1 # [ 0.000000] Normal empty worker1 # [ 0.000000] Device empty client # [ 0.005544] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.000000] Movable zone start for each node client # [ 0.005547] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.000000] Early memory node ranges client # [ 0.005550] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005555] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.005595] alternatives: applying system-wide alternatives worker1 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.008530] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] psci: Trusted OS migration not required client # [ 0.008713] Memory: 1405876K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.009074] devtmpfs: initialized client # [ 0.010923] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.010946] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.011118] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.011122] 0 pages in range for non-PLT usage worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.011123] 508272 pages in range for PLT usage worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.011218] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.011985] DMI not present or invalid. worker1 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.015615] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.017995] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.018188] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.018389] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.018409] audit: initializing netlink subsys (disabled) client # [ 0.018950] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.018952] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hmsqrrazdjx0ansfssikbgws6gj8896v-nixos-system-worker1-test/init regInfo=/nix/store/8vnpypswqgsjck7hj1wpw5i0fw2n1v8l-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.018955] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.018958] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/8vnpypswqgsjck7hj1wpw5i0fw2n1v8l-closure-info/registration", will be passed to user space. client # [ 0.018963] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.018990] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 client # [ 0.018999] cpuidle: using governor ladder worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.019005] cpuidle: using governor menu worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.019210] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.019227] ASID allocator initialised with 65536 entries worker1 # [ 0.000000] software IO TLB: area num 1. client # [ 0.020516] Serial: AMBA PL011 UART driver worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.026020] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.026187] printk: console [ttyAMA0] enabled worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.150830] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.150849] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.150854] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.150858] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.150863] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.150867] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.150872] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.150876] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.158320] fbcon: Taking over console worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.158335] ACPI: Interpreter disabled. worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.160211] iommu: Default domain type: Translated worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.160222] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.167366] SCSI subsystem initialized worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.168058] usbcore: registered new interface driver usbfs worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.168089] usbcore: registered new interface driver hub worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] KASLR enabled worker2 # [ 0.000000] random: crng init done worker1 # [ 0.000032] arm-pv: using stolen time PV client # [ 0.168104] usbcore: registered new device driver usb worker2 # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.168395] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000516] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000671] Console: colour dummy device 80x25 client # [ 0.168401] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.168411] PTP clock support registered worker1 # [ 0.000678] printk: legacy console [tty0] enabled client # [ 0.168458] EDAC MC: Ver: 3.0.0 worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] KASLR enabled client # [ 0.173042] scmi_core: SCMI protocol bus registered worker1 # [ 0.000877] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.000000] Zone ranges: lb # [ 0.000000] random: crng init done worker1 # [ 0.000884] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.173986] FPGA manager framework worker2 # [ 0.000000] DMA32 empty worker1 # [ 0.000959] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] Normal empty client # [ 0.174908] vgaarb: loaded worker1 # [ 0.001124] landlock: Up and running. worker2 # [ 0.000000] Device empty lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.001127] Yama: becoming mindful. worker2 # [ 0.000000] Movable zone start for each node client # [ 0.175547] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.001687] LSM support for eBPF active worker2 # [ 0.000000] Early memory node ranges lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.001836] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] Zone ranges: worker1 # [ 0.001869] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.003729] rcu: Hierarchical SRCU implementation. worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 lb # [ 0.000000] DMA32 empty lb # [ 0.000000] Normal empty worker1 # [ 0.003734] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] Device empty worker1 # [ 0.005013] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] Movable zone start for each node worker1 # [ 0.005105] EFI services will not be available. lb # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.005215] smp: Bringing up secondary CPUs ... worker2 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005223] smp: Brought up 1 node, 1 CPU worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.005226] SMP: Total of 1 processors activated. lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.005232] CPU: All CPU(s) started at EL1 worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.181977] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.005242] CPU features: detected: Branch Target Identification lb # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.182014] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.005247] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.183597] netfs: FS-Cache loaded worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.005250] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.183716] pnp: PnP ACPI: disabled lb # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.005253] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.005257] CPU features: detected: Common not Private translations worker2 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.005260] CPU features: detected: CRC32 instructions lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.005263] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.187427] NET: Registered PF_INET protocol family lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.005266] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.005269] CPU features: detected: Data independent timing control (DIT) lb # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.005272] CPU features: detected: E0PD lb # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.005275] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.005278] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.005282] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bw3mqal5y2l6sysyr6dir665462635vh-nixos-system-worker2-test/init regInfo=/nix/store/xmy09rgpfkkv98has2xhagcrihy5dqwl-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.188280] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.005285] CPU features: detected: Fine Grained Traps worker1 # [ 0.005288] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/xmy09rgpfkkv98has2xhagcrihy5dqwl-closure-info/registration", will be passed to user space. worker1 # [ 0.005293] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.005296] CPU features: detected: LSE atomic instructions worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.005299] CPU features: detected: Privileged Access Never lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hw6sjiya3fqcxqx5sjqhqri1ayky0v1l-nixos-system-lb-test/init regInfo=/nix/store/icv2rkyzy36dhd3gwpgrrp51zhv2f8w0-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.005302] CPU features: detected: PMUv3 worker1 # [ 0.005305] CPU features: detected: RAS Extension Support worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.005308] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.005310] CPU features: detected: Random Number Generator lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/icv2rkyzy36dhd3gwpgrrp51zhv2f8w0-closure-info/registration", will be passed to user space. worker1 # [ 0.005313] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.005316] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.005319] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.005323] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Policy zone: DMA worker1 # [ 0.005362] alternatives: applying system-wide alternatives lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.008318] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.008511] Memory: 1405624K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.008891] devtmpfs: initialized worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.010711] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.010733] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.010937] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.010942] 0 pages in range for non-PLT usage lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.010943] 508272 pages in range for PLT usage worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.011067] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.011904] DMI not present or invalid. lb # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.015269] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.017504] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.017696] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.017896] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.017919] audit: initializing netlink subsys (disabled) lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.018475] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.018477] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.018480] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.018483] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.018489] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.018521] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.018530] cpuidle: using governor ladder worker1 # [ 0.018536] cpuidle: using governor menu lb # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.018744] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.018759] ASID allocator initialised with 65536 entries worker1 # [ 0.020029] Serial: AMBA PL011 UART driver lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.025406] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.025581] printk: console [ttyAMA0] enabled lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.216349] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.216388] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000029] arm-pv: using stolen time PV client # [ 0.216413] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.216480] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.000378] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000565] Console: colour dummy device 80x25 client # [ 0.216623] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.000572] printk: legacy console [tty0] enabled client # [ 0.216701] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.144141] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.216735] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.000795] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.144158] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.000802] pid_max: default: 32768 minimum: 301 lb # [ 0.000030] arm-pv: using stolen time PV client # [ 0.216770] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.144163] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.000881] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.216858] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.144168] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.001016] landlock: Up and running. lb # [ 0.000407] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.216877] NET: Registered PF_XDP protocol family worker2 # [ 0.001018] Yama: becoming mindful. lb # [ 0.000572] Console: colour dummy device 80x25 worker1 # [ 0.144173] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.216891] PCI: CLS 0 bytes, default 64 worker2 # [ 0.001450] LSM support for eBPF active lb # [ 0.000580] printk: legacy console [tty0] enabled client # [ 0.217126] Trying to unpack rootfs image as initramfs... worker1 # [ 0.144177] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.001583] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.144181] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.000794] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.001615] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.144186] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.000801] pid_max: default: 32768 minimum: 301 worker2 # [ 0.003485] rcu: Hierarchical SRCU implementation. lb # [ 0.000874] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.003490] rcu: Max phase no-delay instances is 1000. lb # [ 0.001012] landlock: Up and running. worker2 # [ 0.004746] fsl-mc MSI: its@8080000 domain created lb # [ 0.001015] Yama: becoming mindful. worker2 # [ 0.004835] EFI services will not be available. lb # [ 0.001470] LSM support for eBPF active worker1 # [ 0.151251] fbcon: Taking over console worker2 # [ 0.004906] smp: Bringing up secondary CPUs ... worker1 # [ 0.151267] ACPI: Interpreter disabled. worker2 # [ 0.004914] smp: Brought up 1 node, 1 CPU lb # [ 0.001599] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.004917] SMP: Total of 1 processors activated. lb # [ 0.001632] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.004922] CPU: All CPU(s) started at EL1 lb # [ 0.003581] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004933] CPU features: detected: Branch Target Identification lb # [ 0.003586] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.004938] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.233065] kvm [1]: HYP mode not available lb # [ 0.004889] fsl-mc MSI: its@8080000 domain created lb # [ 0.004987] EFI services will not be available. worker2 # [ 0.004941] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.005073] smp: Bringing up secondary CPUs ... lb # [ 0.005081] smp: Brought up 1 node, 1 CPU worker2 # [ 0.004945] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.005084] SMP: Total of 1 processors activated. lb # [ 0.005090] CPU: All CPU(s) started at EL1 worker2 # [ 0.004948] CPU features: detected: Common not Private translations worker2 # [ 0.004952] CPU features: detected: CRC32 instructions lb # [ 0.005100] CPU features: detected: Branch Target Identification lb # [ 0.005103] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.004954] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.004958] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.160263] iommu: Default domain type: Translated lb # [ 0.005106] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.004961] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.160273] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.005110] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.004964] CPU features: detected: E0PD worker1 # [ 0.160634] SCSI subsystem initialized lb # [ 0.005114] CPU features: detected: Common not Private translations worker2 # [ 0.004966] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.005117] CPU features: detected: CRC32 instructions worker1 # [ 0.162623] usbcore: registered new interface driver usbfs worker2 # [ 0.004970] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.162656] usbcore: registered new interface driver hub lb # [ 0.005120] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.004973] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.162672] usbcore: registered new device driver usb lb # [ 0.005124] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.162970] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.004977] CPU features: detected: Fine Grained Traps lb # [ 0.005127] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.004981] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.162976] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.005130] CPU features: detected: E0PD worker1 # [ 0.162987] PTP clock support registered worker2 # [ 0.004986] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.163037] EDAC MC: Ver: 3.0.0 lb # [ 0.005133] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004989] CPU features: detected: LSE atomic instructions worker1 # [ 0.167489] scmi_core: SCMI protocol bus registered lb # [ 0.005136] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.004992] CPU features: detected: Privileged Access Never worker2 # [ 0.004994] CPU features: detected: PMUv3 lb # [ 0.005140] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.168457] FPGA manager framework worker2 # [ 0.004997] CPU features: detected: RAS Extension Support lb # [ 0.005143] CPU features: detected: Fine Grained Traps worker1 # [ 0.169382] vgaarb: loaded worker2 # [ 0.005000] CPU features: detected: RASv1p1 Extension Support lb # [ 0.005147] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.170001] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.005003] CPU features: detected: Random Number Generator lb # [ 0.005152] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.005005] CPU features: detected: Speculation barrier (SB) lb # [ 0.005155] CPU features: detected: LSE atomic instructions worker2 # [ 0.005008] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.005158] CPU features: detected: Privileged Access Never lb # [ 0.005161] CPU features: detected: PMUv3 lb # [ 0.005164] CPU features: detected: RAS Extension Support lb # [ 0.005167] CPU features: detected: RASv1p1 Extension Support lb # [ 0.005169] CPU features: detected: Random Number Generator lb # [ 0.005172] CPU features: detected: Speculation barrier (SB) lb # [ 0.005175] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.176036] VFS: Disk quotas dquot_6.6.0 lb # [ 0.005178] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.176078] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.005183] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.177742] netfs: FS-Cache loaded worker1 # [ 0.177881] pnp: PnP ACPI: disabled lb # [ 0.005221] alternatives: applying system-wide alternatives lb # [ 0.008190] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.005011] CPU features: detected: TLB range maintenance instructions lb # [ 0.008348] Memory: 1405680K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) lb # [ 0.008723] devtmpfs: initialized worker2 # [ 0.005016] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.181729] NET: Registered PF_INET protocol family worker2 # [ 0.005053] alternatives: applying system-wide alternatives worker2 # [ 0.008009] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.181993] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.008209] Memory: 1405680K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) worker2 # [ 0.008560] devtmpfs: initialized worker2 # [ 0.010339] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.010361] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.010535] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.010539] 0 pages in range for non-PLT usage worker2 # [ 0.010540] 508272 pages in range for PLT usage worker2 # [ 0.010645] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.011403] DMI not present or invalid. lb # [ 0.010529] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.014448] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.010552] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.016677] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.010727] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.016847] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.010731] 0 pages in range for non-PLT usage lb # [ 0.010732] 508272 pages in range for PLT usage worker2 # [ 0.017045] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.010836] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.017068] audit: initializing netlink subsys (disabled) lb # [ 0.011643] DMI not present or invalid. worker2 # [ 0.017645] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.014776] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.017647] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.017249] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.017651] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.017447] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.017654] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.017643] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.017659] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.017665] audit: initializing netlink subsys (disabled) worker2 # [ 0.017691] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.018258] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.017699] cpuidle: using governor ladder worker2 # [ 0.017706] cpuidle: using governor menu lb # [ 0.018260] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.017894] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.018264] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.017910] ASID allocator initialised with 65536 entries lb # [ 0.018266] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.019145] Serial: AMBA PL011 UART driver lb # [ 0.018271] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.024402] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.018302] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.024550] printk: console [ttyAMA0] enabled lb # [ 0.018310] cpuidle: using governor ladder lb # [ 0.018316] cpuidle: using governor menu lb # [ 0.018493] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.145672] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.018507] ASID allocator initialised with 65536 entries worker2 # [ 0.145695] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.019727] Serial: AMBA PL011 UART driver worker2 # [ 0.145700] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.025013] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.145705] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.025163] printk: console [ttyAMA0] enabled worker2 # [ 0.145709] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.145714] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.146651] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.145718] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.146669] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.145723] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.146674] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.146679] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.146683] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.153303] fbcon: Taking over console lb # [ 0.146688] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.153317] ACPI: Interpreter disabled. lb # [ 0.146692] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker2 # [ 0.155208] iommu: Default domain type: Translated lb # [ 0.146696] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.155219] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.156923] SCSI subsystem initialized lb # [ 0.154257] fbcon: Taking over console lb # [ 0.154271] ACPI: Interpreter disabled. lb # [ 0.156215] iommu: Default domain type: Translated worker1 # [ 0.210944] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.156225] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.210985] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.157938] SCSI subsystem initialized worker1 # [ 0.211012] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.211085] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.211228] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.211303] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.211336] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.164906] usbcore: registered new interface driver usbfs worker1 # [ 0.211372] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.164948] usbcore: registered new interface driver hub worker1 # [ 0.211460] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.164964] usbcore: registered new device driver usb lb # [ 0.163061] usbcore: registered new interface driver usbfs worker1 # [ 0.211513] NET: Registered PF_XDP protocol family worker2 # [ 0.165243] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.163094] usbcore: registered new interface driver hub worker1 # [ 0.211528] PCI: CLS 0 bytes, default 64 lb # [ 0.163109] usbcore: registered new device driver usb worker1 # [ 0.211760] Trying to unpack rootfs image as initramfs... worker2 # [ 0.165250] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.163377] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.165260] PTP clock support registered worker2 # [ 0.165308] EDAC MC: Ver: 3.0.0 lb # [ 0.163383] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.169907] scmi_core: SCMI protocol bus registered lb # [ 0.163393] PTP clock support registered lb # [ 0.163441] EDAC MC: Ver: 3.0.0 worker2 # [ 0.170934] FPGA manager framework lb # [ 0.168063] scmi_core: SCMI protocol bus registered worker2 # [ 0.171935] vgaarb: loaded lb # [ 0.169038] FPGA manager framework lb # [ 0.169966] vgaarb: loaded worker2 # [ 0.172573] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.173199] VFS: Disk quotas dquot_6.6.0 lb # [ 0.170579] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.173232] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.227543] kvm [1]: HYP mode not available worker2 # [ 0.175677] netfs: FS-Cache loaded worker2 # [ 0.175800] pnp: PnP ACPI: disabled lb # [ 0.176593] VFS: Disk quotas dquot_6.6.0 lb # [ 0.176634] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.324053] Initialise system trusted keyrings lb # [ 0.178258] netfs: FS-Cache loaded lb # [ 0.178360] pnp: PnP ACPI: disabled client # [ 0.324788] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.326051] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.181835] NET: Registered PF_INET protocol family client # [ 0.326821] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.182097] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.182292] NET: Registered PF_INET protocol family lb # [ 0.182554] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.355605] Key type asymmetric registered client # [ 0.355630] Asymmetric key parser 'x509' registered client # [ 0.355700] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.357820] io scheduler mq-deadline registered client # [ 0.357831] io scheduler kyber registered worker2 # [ 0.212826] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.212870] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.211545] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.212900] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.211591] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.212980] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.211618] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.213124] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.211693] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.213204] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.367694] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.211846] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.213239] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.211924] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.213276] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.211959] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.213367] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.211995] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.213396] NET: Registered PF_XDP protocol family client # [ 0.369109] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.213417] PCI: CLS 0 bytes, default 64 lb # [ 0.212083] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.213654] Trying to unpack rootfs image as initramfs... lb # [ 0.212114] NET: Registered PF_XDP protocol family client # [ 0.369524] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.212130] PCI: CLS 0 bytes, default 64 lb # [ 0.212356] Trying to unpack rootfs image as initramfs... client # [ 0.369541] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.369554] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.230268] kvm [1]: HYP mode not available client # [ 0.369563] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.228368] kvm [1]: HYP mode not available client # [ 0.369586] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.369610] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.369688] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.369698] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.369704] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.369709] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.369714] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.369802] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.370251] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.370441] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.370458] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.370488] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.370504] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.370965] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.371152] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.371168] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.371198] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.390859] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.391046] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.391062] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.391092] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.394707] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.394894] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.394909] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.394939] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.394957] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.395411] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.318545] Initialise system trusted keyrings worker1 # [ 0.319277] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.320539] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.321315] 9p: Installing v9fs 9p2000 file system support client # [ 0.407665] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.407709] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.408209] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.408403] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.408433] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.408825] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.409010] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.409297] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.409487] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409518] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409979] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.410166] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.410197] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.341974] Key type asymmetric registered worker1 # [ 0.341998] Asymmetric key parser 'x509' registered client # [ 0.410647] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.410834] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.410864] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.411333] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.411617] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.411634] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.411664] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.412122] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.412306] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.412322] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.350091] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.412352] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.351164] io scheduler mq-deadline registered client # [ 0.412947] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.351175] io scheduler kyber registered client # [ 0.412958] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.412963] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.413008] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned client # [ 0.413055] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.413103] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.413151] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.413198] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.413247] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.413293] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.413340] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.413385] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.362143] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.413456] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.413502] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.413523] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.413545] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.363539] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.413568] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.363953] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.413589] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.363971] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.413611] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.363984] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.413633] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned client # [ 0.413657] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.363992] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.413679] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.364015] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.413701] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.364041] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.413724] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.364116] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.413747] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.364125] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.413780] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.364131] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.413802] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.364137] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.413822] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.364142] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.413843] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned client # [ 0.413864] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.364230] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.413885] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.364660] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.413911] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.364843] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.413920] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.364860] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.413924] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.364890] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.414747] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.319124] Initialise system trusted keyrings worker1 # [ 0.364906] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.319880] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.365354] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.365534] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.324249] Initialise system trusted keyrings lb # [ 0.321143] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.365550] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.325032] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.365580] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.321903] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.326286] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.385064] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.385248] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.327072] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.385264] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.385294] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.385763] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.385944] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.385960] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.385990] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392184] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.392633] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.392813] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.392843] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.393278] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.393458] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.393487] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.393865] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.394050] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.394339] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.394519] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.394548] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.491012] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.394988] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.395171] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.351111] Key type asymmetric registered worker1 # [ 0.395201] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.351136] Asymmetric key parser 'x509' registered worker1 # [ 0.395645] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.351217] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.395826] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.353441] io scheduler mq-deadline registered worker1 # [ 0.395856] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.350647] Key type asymmetric registered worker2 # [ 0.353450] io scheduler kyber registered lb # [ 0.350674] Asymmetric key parser 'x509' registered worker1 # [ 0.396300] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.350744] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.396555] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.352857] io scheduler mq-deadline registered worker1 # [ 0.396573] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.352868] io scheduler kyber registered worker1 # [ 0.396602] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.397045] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.397230] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.397246] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.397275] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.397820] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.500947] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.397831] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.397836] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.397879] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.397924] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.397969] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.362702] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.505370] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.507311] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.364108] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.509340] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.364520] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.364538] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.364563] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.364573] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.364594] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.364618] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.368754] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.364697] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.369425] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.364706] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.369801] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.364723] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.369819] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.364729] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.364735] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.369831] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.519651] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.364790] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.369840] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.521466] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.365222] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.369861] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.523243] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.365406] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.438107] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.369889] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.365423] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.369966] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.365453] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.438173] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.365469] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.369976] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.438222] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.369983] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.365927] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.438269] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.369988] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.366108] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.438316] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.369994] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.366125] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.366154] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.438364] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.370071] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.438483] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.370525] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.370718] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.438533] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.385723] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.370735] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.385909] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.438556] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.385925] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.370764] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.438579] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.385956] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.370781] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.438601] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.371253] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.386437] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.438623] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.371443] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.438645] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.533540] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.371459] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.535475] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.438667] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.371490] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.438696] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.371955] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.438718] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.372140] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.438740] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.372156] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.438763] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.372185] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.438785] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.372663] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.438807] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.372848] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.438828] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.372863] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.438849] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.372893] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.438871] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.372911] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.438892] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.438913] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.373373] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.438942] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.373559] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.438952] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.373589] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.545543] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.438957] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.374048] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.394695] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.439772] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.374235] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.394723] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.374268] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.394754] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.394774] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.374688] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.374874] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.395279] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.395474] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.375146] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.375334] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.395504] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.375364] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.395988] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.375834] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.396175] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.376020] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.396205] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.376050] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.396596] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.376502] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.396777] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.397054] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.397240] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.397270] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.560773] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.397724] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.397909] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.397938] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.563287] msm_serial: driver initialized client # [ 0.563431] SuperH (H)SCI(F) driver initialized lb # [ 0.398386] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.563485] STM32 USART driver initialized lb # [ 0.398570] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.398616] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.399076] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.399351] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.420782] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.399368] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.420829] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.399398] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.421360] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.421679] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.399882] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.483937] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.421696] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.400064] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.400079] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.421726] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.400109] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.422185] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.400696] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.422368] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.422383] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.400707] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.422413] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.400713] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.422988] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.400758] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.423016] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.400803] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.423023] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.400850] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.423069] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.400896] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.423116] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.400942] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.423163] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.400989] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.423210] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.401035] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.423258] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.401081] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.423305] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.401127] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.423352] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.401198] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.423399] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.401244] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.423445] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.494136] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.401266] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.497868] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.423513] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.401288] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.499970] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.423559] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.401309] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.423580] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.401331] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.423601] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.401353] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.423621] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.401375] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.423643] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.401399] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.423663] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.401421] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.423684] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.401443] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.423710] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.401466] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.423732] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.401488] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.423753] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.401509] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.423775] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.401531] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.423797] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.401552] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.423817] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.401574] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.510183] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.423838] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.401595] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.512413] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.423858] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.597726] loop: module loaded lb # [ 0.401617] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.423879] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.401644] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.597977] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.423899] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.401653] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.598806] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.423921] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.401658] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.423946] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.402460] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.423956] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.423960] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.464858] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.604107] megasas: 07.734.00.00-rc1 lb # [ 0.462698] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 58829 usecs client # [ 0.604841] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.522166] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.524527] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.526416] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.528468] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.538591] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.624683] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.624714] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.626294] Using buffer write method client # [ 0.626338] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.486033] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.488158] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.631361] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.631384] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.635768] Using buffer write method client # [ 0.635791] Concatenating MTD devices: client # [ 0.635795] (0): "0.flash" lb # [ 0.489135] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.635799] (1): "0.flash" client # [ 0.635810] into device "0.flash" worker1 # [ 0.555271] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.492100] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.494077] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.499178] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.558221] msm_serial: driver initialized worker1 # [ 0.558382] SuperH (H)SCI(F) driver initialized worker1 # [ 0.558436] STM32 USART driver initialized lb # [ 0.504035] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.506179] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.509192] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.512866] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.514881] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.517664] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.520191] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.522058] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.524698] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.526901] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.588527] loop: module loaded worker1 # [ 0.588703] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.589500] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.532191] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.534127] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.536777] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.539326] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.594608] megasas: 07.734.00.00-rc1 worker1 # [ 0.595351] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.543123] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.545567] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.557807] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.560425] msm_serial: driver initialized worker2 # [ 0.560571] SuperH (H)SCI(F) driver initialized lb # [ 0.559821] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.619258] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.619294] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.620981] Using buffer write method lb # [ 0.562533] msm_serial: driver initialized worker1 # [ 0.621048] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.561724] STM32 USART driver initialized lb # [ 0.563409] SuperH (H)SCI(F) driver initialized lb # [ 0.563471] STM32 USART driver initialized worker1 # [ 0.626186] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.626257] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.628026] Using buffer write method worker1 # [ 0.628052] Concatenating MTD devices: worker1 # [ 0.628056] (0): "0.flash" worker1 # [ 0.628060] (1): "0.flash" worker1 # [ 0.628070] into device "0.flash" worker2 # [ 0.594794] loop: module loaded worker2 # [ 0.594980] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.593381] loop: module loaded worker2 # [ 0.595995] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.593570] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.594359] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.601217] megasas: 07.734.00.00-rc1 worker2 # [ 0.601929] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.599157] megasas: 07.734.00.00-rc1 lb # [ 0.599864] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.612616] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.612643] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.614322] Using buffer write method lb # [ 0.611749] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.611780] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.614385] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.613381] Using buffer write method lb # [ 0.613444] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.620280] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.620305] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.624856] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.630431] Using buffer write method worker2 # [ 0.630468] Concatenating MTD devices: lb # [ 0.624886] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.630472] (0): "0.flash" worker2 # [ 0.630476] (1): "0.flash" worker2 # [ 0.630484] into device "0.flash" lb # [ 0.634618] Using buffer write method lb # [ 0.634644] Concatenating MTD devices: lb # [ 0.634648] (0): "0.flash" lb # [ 0.634656] (1): "0.flash" lb # [ 0.634660] into device "0.flash" client # [ 0.867781] Freeing initrd memory: 26972K client # [ 0.873784] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.877490] thunder_xcv, ver 1.0 client # [ 0.877532] thunder_bgx, ver 1.0 client # [ 0.877555] nicpf, ver 1.0 client # [ 0.878124] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.878131] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.878153] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.878161] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.878189] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.878194] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.878216] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.878222] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.878355] sky2: driver version 1.30 client # [ 0.880241] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.880266] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.880517] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.890304] usbcore: registered new interface driver usb-storage client # [ 0.890354] usbcore: registered new interface driver usbserial_generic client # [ 0.890367] usbserial: USB Serial support registered for generic client # [ 0.890948] hv_vmbus: registering driver hyperv_keyboard client # [ 0.891605] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.891902] hub 1-0:1.0: USB hub found client # [ 0.891922] hub 1-0:1.0: 6 ports detected client # [ 0.897080] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.897109] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:24:59 UTC (1790076299) client # [ 0.897421] i2c_dev: i2c /dev entries driver client # [ 0.902812] sdhci: Secure Digital Host Controller Interface driver client # [ 0.902826] sdhci: Copyright(c) Pierre Ossman client # [ 0.903085] Synopsys Designware Multimedia Card Interface Driver client # [ 0.903450] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.907710] hid: raw HID events driver (C) Jiri Kosina client # [ 0.907951] usbcore: registered new interface driver usbhid client # [ 0.907958] usbhid: USB HID core driver client # [ 0.910858] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.913473] drop_monitor: Initializing network drop monitor service client # [ 0.913631] NET: Registered PF_INET6 protocol family client # [ 0.915519] Segment Routing with IPv6 client # [ 0.916202] In-situ OAM (IOAM) with IPv6 client # [ 0.916240] NET: Registered PF_PACKET protocol family client # [ 0.917388] 9pnet: Installing 9P2000 support client # [ 0.917435] Key type dns_resolver registered client # [ 0.924124] registered taskstats version 1 client # [ 0.924281] Loading compiled-in X.509 certificates client # [ 0.932949] Demotion targets for Node 0: null client # [ 0.933052] Key type .fscrypt registered client # [ 0.933058] Key type fscrypt-provisioning registered client # [ 0.933149] ima: No TPM chip found, activating TPM-bypass! client # [ 0.933168] ima: Allocated hash algorithm: sha1 client # [ 0.933190] ima: No architecture policies found client # [ 0.937255] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.955064] clk: Disabling unused clocks client # [ 0.955094] PM: genpd: Disabling unused power domains worker1 # [ 0.872968] Freeing initrd memory: 26976K client # [ 0.959254] Freeing unused kernel memory: 4736K client # [ 0.959452] Run /init as init process worker1 # [ 0.879223] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.883027] thunder_xcv, ver 1.0 worker1 # [ 0.883073] thunder_bgx, ver 1.0 worker1 # [ 0.883095] nicpf, ver 1.0 worker1 # [ 0.883725] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.883732] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.883757] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.973796] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.883765] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.883794] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.883799] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.883823] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.883829] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.883965] sky2: driver version 1.30 worker1 # [ 0.885536] usbcore: registered new interface driver usb-storage worker1 # [ 0.885619] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.885634] usbserial: USB Serial support registered for generic worker1 # [ 0.886727] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.886752] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.887025] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.898539] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker1 # [ 0.898863] hub 1-0:1.0: USB hub found worker1 # [ 0.898879] hub 1-0:1.0: 6 ports detected worker1 # [ 0.901164] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.902806] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.902832] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:24:59 UTC (1790076299) worker1 # [ 0.903138] i2c_dev: i2c /dev entries driver worker1 # [ 0.908484] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.908495] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.908759] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.909130] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.913346] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.913623] usbcore: registered new interface driver usbhid worker1 # [ 0.913629] usbhid: USB HID core driver worker1 # [ 0.916489] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.919158] drop_monitor: Initializing network drop monitor service worker1 # [ 0.919321] NET: Registered PF_INET6 protocol family worker1 # [ 0.921194] Segment Routing with IPv6 worker1 # [ 0.921213] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.921240] NET: Registered PF_PACKET protocol family worker1 # [ 0.923076] 9pnet: Installing 9P2000 support worker1 # [ 0.923126] Key type dns_resolver registered worker1 # [ 0.929900] registered taskstats version 1 worker2 # [ 0.872259] Freeing initrd memory: 26944K worker1 # [ 0.930575] Loading compiled-in X.509 certificates worker2 # [ 0.878299] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.938783] Demotion targets for Node 0: null worker1 # [ 0.938893] Key type .fscrypt registered worker1 # [ 0.938901] Key type fscrypt-provisioning registered worker1 # [ 0.939000] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.939019] ima: Allocated hash algorithm: sha1 worker1 # [ 0.939041] ima: No architecture policies found worker1 # [ 0.943011] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.881991] thunder_xcv, ver 1.0 worker2 # [ 0.882033] thunder_bgx, ver 1.0 worker2 # [ 0.882056] nicpf, ver 1.0 worker2 # [ 0.882627] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.882634] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.883015] Freeing initrd memory: 26948K worker2 # [ 0.882659] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.882667] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.882696] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.882702] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.882725] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.882730] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.882862] sky2: driver version 1.30 lb # [ 0.889040] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.884460] usbcore: registered new interface driver usb-storage worker2 # [ 0.884543] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.884557] usbserial: USB Serial support registered for generic worker2 # [ 0.885344] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.885370] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.885648] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.892832] thunder_xcv, ver 1.0 worker2 # [ 0.897198] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.892879] thunder_bgx, ver 1.0 worker2 # [ 0.897505] hub 1-0:1.0: USB hub found lb # [ 0.892901] nicpf, ver 1.0 worker2 # [ 0.897531] hub 1-0:1.0: 6 ports detected lb # [ 0.893480] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.893487] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.893512] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.900112] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.893519] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.893552] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.893557] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.901668] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.893580] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.961257] clk: Disabling unused clocks lb # [ 0.893585] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.961285] PM: genpd: Disabling unused power domains worker2 # [ 0.901698] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:24:59 UTC (1790076299) lb # [ 0.893719] sky2: driver version 1.30 worker2 # [ 0.902006] i2c_dev: i2c /dev entries driver lb # [ 0.902425] usbcore: registered new interface driver usb-storage worker1 # [ 0.965383] Freeing unused kernel memory: 4736K lb # [ 0.902550] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.965606] Run /init as init process lb # [ 0.902564] usbserial: USB Serial support registered for generic worker2 # [ 0.907298] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.903211] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.907307] sdhci: Copyright(c) Pierre Ossman lb # [ 0.903234] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.907568] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.903397] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.907935] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.908670] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.912151] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.912385] usbcore: registered new interface driver usbhid worker2 # [ 0.912391] usbhid: USB HID core driver lb # [ 0.910246] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.910276] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:24:59 UTC (1790076299) lb # [ 0.912314] i2c_dev: i2c /dev entries driver worker2 # [ 0.915282] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.917899] drop_monitor: Initializing network drop monitor service worker2 # [ 0.918089] NET: Registered PF_INET6 protocol family worker2 # [ 0.920030] Segment Routing with IPv6 worker2 # [ 0.920049] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.920077] NET: Registered PF_PACKET protocol family worker1 # [ 0.980201] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.916070] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.916081] sdhci: Copyright(c) Pierre Ossman lb # [ 0.916352] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.916723] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.918280] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.921886] 9pnet: Installing 9P2000 support worker2 # [ 0.921931] Key type dns_resolver registered lb # [ 0.918510] usbcore: registered new interface driver usbhid lb # [ 0.918516] usbhid: USB HID core driver lb # [ 0.923114] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.924240] hub 1-0:1.0: USB hub found lb # [ 0.924754] hub 1-0:1.0: 6 ports detected worker2 # [ 0.928558] registered taskstats version 1 lb # [ 0.926708] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.929243] Loading compiled-in X.509 certificates lb # [ 0.928243] drop_monitor: Initializing network drop monitor service lb # [ 0.928367] NET: Registered PF_INET6 protocol family lb # [ 0.931403] Segment Routing with IPv6 lb # [ 0.931420] In-situ OAM (IOAM) with IPv6 lb # [ 0.931450] NET: Registered PF_PACKET protocol family lb # [ 0.933112] 9pnet: Installing 9P2000 support worker2 # [ 0.937359] Demotion targets for Node 0: null worker2 # [ 0.937471] Key type .fscrypt registered lb # [ 0.933162] Key type dns_resolver registered worker2 # [ 0.937478] Key type fscrypt-provisioning registered worker2 # [ 0.937573] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.937591] ima: Allocated hash algorithm: sha1 worker2 # [ 0.937613] ima: No architecture policies found worker2 # [ 0.941596] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.940031] registered taskstats version 1 lb # [ 0.940176] Loading compiled-in X.509 certificates lb # [ 0.949085] Demotion targets for Node 0: null lb # [ 0.949188] Key type .fscrypt registered lb # [ 0.949194] Key type fscrypt-provisioning registered lb # [ 0.949292] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.949311] ima: Allocated hash algorithm: sha1 lb # [ 0.949333] ima: No architecture policies found lb # [ 0.953419] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.959835] clk: Disabling unused clocks worker2 # [ 0.959859] PM: genpd: Disabling unused power domains worker2 # [ 0.963980] Freeing unused kernel memory: 4736K worker2 # [ 0.964200] Run /init as init process lb # [ 0.971697] clk: Disabling unused clocks lb # [ 0.971720] PM: genpd: Disabling unused power domains worker2 # [ 0.978767] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.975900] Freeing unused kernel memory: 4736K lb # [ 0.976099] Run /init as init process lb # [ 0.992830] systemd[1]: Successfully made /usr/ read-only. client # [ 1.139623] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.146090] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.144614] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.289659] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.170668] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.308621] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.320920] systemd[1]: Detected virtualization qemu. client # [ 1.322939] systemd[1]: Detected architecture arm64. client # [ 1.324934] systemd[1]: Running in initrd. client # [ 1.327677] systemd[1]: Initializing machine ID from random generator. client # [ 1.330521] systemd[1]: Hostname set to . client # [ 1.375837] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.296001] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.315136] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.321157] systemd[1]: Detected virtualization qemu. worker1 # [ 1.322093] systemd[1]: Detected architecture arm64. worker1 # [ 1.322940] systemd[1]: Running in initrd. worker1 # [ 1.324170] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.325424] systemd[1]: Hostname set to . worker2 # [ 1.297140] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.378246] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.313408] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.325665] systemd[1]: Detected virtualization qemu. worker2 # [ 1.327664] systemd[1]: Detected architecture arm64. worker2 # [ 1.329624] systemd[1]: Running in initrd. lb # [ 1.323109] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.332211] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.335336] systemd[1]: Hostname set to . lb # [ 1.329343] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.341811] systemd[1]: Detected virtualization qemu. lb # [ 1.341910] systemd[1]: Detected architecture arm64. lb # [ 1.341935] systemd[1]: Running in initrd. lb # [ 1.348519] systemd[1]: Initializing machine ID from random generator. lb # [ 1.351392] systemd[1]: Hostname set to . client # [ 1.499598] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.380876] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.406836] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.498092] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.642628] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.504673] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.662321] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.526621] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.671739] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.616791] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.654659] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.658970] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 1.756696] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.766366] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.767666] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.767713] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.767745] systemd[1]: Reached target Path Units. client # [ 1.767768] systemd[1]: Reached target Slice Units. client # [ 1.767789] systemd[1]: Reached target Swaps. client # [ 1.767807] systemd[1]: Reached target Timer Units. client # [ 1.768019] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.768179] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.768326] systemd[1]: Listening on Journal Sockets. client # [ 1.768459] systemd[1]: Listening on udev Control Socket. client # [ 1.768582] systemd[1]: Listening on udev Kernel Socket. client # [ 1.768603] systemd[1]: Reached target Socket Units. client # [ 1.780936] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.782085] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.645313] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.791741] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.799886] systemd[1]: Starting Journal Service... lb # [ 1.658175] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.723912] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.665151] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.670281] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.732373] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.733582] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.735088] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.735130] systemd[1]: Reached target Path Units. worker1 # [ 1.735154] systemd[1]: Reached target Slice Units. worker1 # [ 1.735175] systemd[1]: Reached target Swaps. client # [ 1.823940] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.735197] systemd[1]: Reached target Timer Units. client # [ 1.824921] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.735395] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.735550] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.735691] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.735828] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.735944] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.735967] systemd[1]: Reached target Socket Units. lb # [ 1.681274] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.746936] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.748184] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.686453] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.755753] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.846021] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.766294] systemd[1]: Starting Journal Service... client # [ 1.847171] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.867749] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.869327] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.790176] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.791172] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.884003] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.809637] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.751202] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.818988] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 1.762893] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.764088] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.765415] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.766427] systemd[1]: Reached target Path Units. worker2 # [ 1.767227] systemd[1]: Reached target Slice Units. worker2 # [ 1.768025] systemd[1]: Reached target Swaps. worker2 # [ 1.768807] systemd[1]: Reached target Timer Units. worker2 # [ 1.769772] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.766977] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.771075] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.772166] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.773169] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.830250] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.915193] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.830970] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.774157] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.775038] systemd[1]: Reached target Socket Units. worker2 # [ 1.777407] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.920234] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.778538] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.776970] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.922754] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.778162] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.779519] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.780557] systemd[1]: Reached target Path Units. lb # [ 1.781341] systemd[1]: Reached target Slice Units. lb # [ 1.782133] systemd[1]: Reached target Swaps. lb # [ 1.782896] systemd[1]: Reached target Timer Units. worker2 # [ 1.785932] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.783894] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.842446] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.785114] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.786199] systemd[1]: Listening on Journal Sockets. client # [ 1.931773] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.787224] systemd[1]: Listening on udev Control Socket. lb # [ 1.787356] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.787379] systemd[1]: Reached target Socket Units. lb # [ 1.791484] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.792644] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.796854] systemd[1]: Starting Journal Service... client # [ 1.940690] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.940931] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.940941] [drm] features: -context_init client # [ 1.941658] [drm] number of scanouts: 1 client # [ 1.941675] [drm] number of cap sets: 0 lb # [ 1.800107] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.810874] systemd[1]: Starting Journal Service... worker2 # [ 1.821217] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.878633] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.822242] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.881146] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.968148] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 1.968486] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.968515] systemd[1]: Reached target Local File Systems. worker1 # [ 1.889277] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.972439] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.976950] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.976973] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.835028] systemd[1]: Starting Load Kernel Modules... lb # [ 1.836097] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.898075] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.845397] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.903986] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.904228] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.904238] [drm] features: -context_init worker1 # [ 1.904917] [drm] number of scanouts: 1 worker1 # [ 1.904934] [drm] number of cap sets: 0 worker2 # [ 1.850690] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.854755] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.860714] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.861464] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.999837] Console: switching to colour frame buffer device 160x50 lb # [ 1.864755] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.871194] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.871890] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.876775] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.930349] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.930367] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 1.938863] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.939225] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.939254] systemd[1]: Reached target Local File Systems. client # [ 2.028186] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 1.944697] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.891005] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.896698] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 2.040335] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.898435] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.900651] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 2.033831] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 2.052099] systemd[1]: Starting Apply Kernel Variables... client # [ 2.052304] systemd[1]: Started Journal Service. worker2 # [ 1.913980] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.914220] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.914230] [drm] features: -context_init worker2 # [ 1.914943] [drm] number of scanouts: 1 client # [ 2.044283] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker2 # [ 1.914961] [drm] number of cap sets: 0 client # [ 2.045448] systemd-modules-load[74]: Module 'virtio_console' is built in client # [ 2.046579] systemd-modules-load[74]: Inserted module 'virtio_gpu' client # [ 2.047569] systemd-modules-load[74]: Module 'virtio_rng' is built in lb # [ 1.922724] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.925681] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.986098] Console: switching to colour frame buffer device 160x50 worker1 # [ 1.986770] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 1.933227] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.935736] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 2.064319] systemd-udevd[79]: Using default interface naming scheme 'v261'. lb # [ 1.935032] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 2.070025] systemd[1]: Starting Create System Files and Directories... worker1 # [ 2.002937] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.940366] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.940388] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.944594] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.944840] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.944850] [drm] features: -context_init lb # [ 1.945575] [drm] number of scanouts: 1 lb # [ 1.945594] [drm] number of cap sets: 0 worker1 # [ 2.010369] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.004303] systemd-modules-load[74]: Inserted module 'dm_mod' worker1 # [ 2.005370] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker1 # [ 2.006448] systemd-modules-load[74]: Module 'virtio_console' is built in worker1 # [ 2.007518] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 2.026247] systemd[1]: Started Journal Service. worker1 # [ 2.016510] systemd-modules-load[74]: Module 'virtio_rng' is built in worker2 # [ 1.964887] Console: switching to colour frame buffer device 160x50 lb # [ 1.966996] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.967019] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.975667] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.976065] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.108709] systemd[1]: Finished Create System Files and Directories. lb # [ 1.976092] systemd[1]: Reached target Local File Systems. worker1 # [ 2.028633] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 2.113525] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.989224] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.989585] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 2.036345] systemd[1]: Starting Create System Files and Directories... worker2 # [ 1.989610] systemd[1]: Reached target Local File Systems. worker2 # [ 1.993587] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.987073] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.125591] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 1.997295] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.053869] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.009824] systemd[1]: Finished Load Kernel Modules. lb # [ 2.000277] Console: switching to colour frame buffer device 160x50 worker2 # [ 2.005558] systemd-modules-load[74]: Inserted module 'dm_mod' worker2 # [ 2.008219] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker2 # [ 2.011060] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 2.023159] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.077040] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.024348] systemd-modules-load[74]: Inserted module 'virtio_gpu' lb # [ 2.039238] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.087437] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.041929] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.042133] systemd[1]: Started Journal Service. worker2 # [ 2.029362] systemd-modules-load[74]: Module 'virtio_rng' is built in lb # [ 2.042939] systemd[1]: Starting Apply Kernel Variables... client # [ 2.184102] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.048201] systemd[1]: Starting Create System Files and Directories... lb # [ 2.048338] systemd-modules-load[74]: Inserted module 'dm_mod' lb # [ 2.049457] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.050552] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 2.051618] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker2 # [ 2.058202] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.070849] systemd[1]: Started Journal Service. lb # [ 2.064466] systemd-modules-load[74]: Module 'virtio_rng' is built in lb # [ 2.072109] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.083691] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.148153] systemd[1]: Starting Virtual Console Setup... client # [ 2.232443] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.088375] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.235841] systemd[1]: Finished Virtual Console Setup. lb # [ 2.090882] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.092537] systemd-udevd[79]: Using default interface naming scheme 'v261'. lb # [ 2.113278] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.117319] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.130121] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.196489] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.199964] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.168608] systemd[1]: Starting Virtual Console Setup... lb # [ 2.184393] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.224468] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.227781] systemd[1]: Finished Virtual Console Setup. lb # [ 2.236471] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.239732] systemd[1]: Finished Virtual Console Setup. client # [ 2.854717] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.855651] systemd[1]: Reached target System Initialization. client # [ 2.860160] systemd[1]: Reached target Basic System. worker1 # [ 2.811121] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.812097] systemd[1]: Reached target System Initialization. worker1 # [ 2.812940] systemd[1]: Reached target Basic System. worker2 # [ 2.824591] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.825534] systemd[1]: Reached target System Initialization. worker2 # [ 2.826335] systemd[1]: Reached target Basic System. client # [ 2.996129] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.865566] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.866518] systemd[1]: Reached target System Initialization. lb # [ 2.867328] systemd[1]: Reached target Basic System. worker1 # [ 2.944054] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.032876] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.037014] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.977943] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 2.984181] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.085953] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.088917] systemd[1]: Reached target Initrd Root Device. client # [ 3.092363] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 2.997473] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.999850] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.145630] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.003676] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.151219] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.009813] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.076985] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.080942] systemd[1]: Reached target Initrd Root Device. client # [ 3.164233] systemd[1]: Mounting /sysroot... worker1 # [ 3.084076] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.045489] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.050246] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.214651] EXT4-fs (vda): mounted filesystem 029cd0ef-f3c7-49c5-af97-28279bde576d r/w with ordered data mode. Quota mode: none. client # [ 3.205868] systemd[1]: Mounted /sysroot. client # [ 3.207605] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.126539] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.212453] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.131708] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.073056] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.136380] systemd[1]: Mounting /sysroot... worker2 # [ 3.076252] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.080424] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.239696] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.103909] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.248597] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.251775] systemd[1]: Reloading... worker1 # [ 3.184288] EXT4-fs (vda): mounted filesystem 85753b96-c7ba-477e-964a-82432a2541e5 r/w with ordered data mode. Quota mode: none. worker1 # [ 3.175568] systemd[1]: Mounted /sysroot. lb # [ 3.113872] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.177535] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.178783] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.120108] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.129903] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.135234] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.199796] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.206433] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.146997] systemd[1]: Mounting /sysroot... worker1 # [ 3.211004] systemd[1]: Reloading... lb # [ 3.167974] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.173887] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.178855] systemd[1]: Mounting /sysroot... worker2 # [ 3.197601] EXT4-fs (vda): mounted filesystem e2934ba0-53e2-4b9d-9be0-9be4a9393436 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.187501] systemd[1]: Mounted /sysroot. worker2 # [ 3.189872] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.196132] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.229570] EXT4-fs (vda): mounted filesystem 14220a4a-ec47-41b7-9745-bca8042d2bff r/w with ordered data mode. Quota mode: none. lb # [ 3.220605] systemd[1]: Mounted /sysroot. lb # [ 3.222750] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.226816] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.227390] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.232422] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.236250] systemd[1]: Reloading... lb # [ 3.255920] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.261846] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.266371] systemd[1]: Reloading... client # [ 3.459514] systemd[1]: Reloading finished in 211 ms. client # [ 3.487067] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.492237] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.497270] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.498752] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.500124] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.420782] systemd[1]: Reloading finished in 211 ms. worker1 # [ 3.451473] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.455736] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... worker1 # [ 3.462858] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.465369] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.467904] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.442091] systemd[1]: Reloading finished in 210 ms. worker2 # [ 3.472268] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... lb # [ 3.474399] systemd[1]: Reloading finished in 209 ms. worker2 # [ 3.477561] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... worker2 # [ 3.483100] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.485225] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.487374] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.504841] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.508521] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.514714] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.517955] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.520129] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.856718] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.875568] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.881192] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.811202] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.898436] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.826815] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.838902] systemd[1]: Mounting /sysroot/run... client # [ 3.922948] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.870227] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.982635] fuse: init (API version 7.45) client # [ 3.969847] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.990317] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 3.833433] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.991128] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.895878] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 4.006180] virtiofs virtio7: discovered new tag: shared client # [ 4.006942] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.851030] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.854941] systemd[1]: Mounting /sysroot/run... lb # [ 3.855857] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.932454] fuse: init (API version 7.45) client # [ 4.017153] virtiofs virtio8: discovered new tag: xchg client # [ 4.017966] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 3.937450] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.929145] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.868922] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 4.014043] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.872774] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.947138] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.936660] systemd[1]: Mounted /sysroot/run. lb # [ 3.875119] systemd[1]: Mounting /sysroot/run... client # [ 4.026572] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.033876] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.963934] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.964691] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 4.036967] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.893535] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 4.039940] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.975671] virtiofs virtio8: discovered new tag: xchg worker1 # [ 3.976417] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 4.056461] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.057569] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.981700] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.984068] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.986826] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.944164] fuse: init (API version 7.45) worker1 # [ 3.995712] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 3.952059] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 3.938657] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.944080] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.962650] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 4.011970] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 4.013198] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 3.967077] fuse: init (API version 7.45) worker2 # [ 3.959453] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.978562] virtiofs virtio7: discovered new tag: shared worker2 # [ 3.979388] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 3.976099] virtiofs virtio6: discovered new tag: nix-store lb # [ 3.976887] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.987703] virtiofs virtio8: discovered new tag: xchg worker2 # [ 3.988445] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 3.992407] virtiofs virtio7: discovered new tag: shared lb # [ 3.993182] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 3.998304] virtiofs virtio8: discovered new tag: xchg lb # [ 4.007211] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 3.998289] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.006542] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 4.009144] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.014002] systemd[1]: Mounted /sysroot/run. lb # [ 4.013066] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.016151] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.015244] systemd[1]: Mounted /sysroot/run. lb # [ 4.017869] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.033315] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.032186] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.035728] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.043708] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.044931] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.065644] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.066764] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.067998] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.118125] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.118970] systemd[1]: Reached target Initrd File Systems. lb # [ 4.121035] systemd[1]: Starting Find NixOS closure... lb # [ 4.131132] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.164978] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.178112] systemd[1]: Finished Find NixOS closure. lb # [ 4.181000] systemd[1]: Reached target Initrd Default Target. lb # [ 4.184703] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.219640] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.224536] systemd[1]: Stopped target Basic System. lb # [ 4.225296] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.226072] systemd[1]: Stopped target Path Units. lb # [ 4.226776] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.227868] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.236451] systemd[1]: Stopped target Slice Units. lb # [ 4.241940] systemd[1]: Stopped target Socket Units. lb # [ 4.243754] systemd[1]: Stopped target System Initialization. lb # [ 4.245353] systemd[1]: Stopped target Swaps. client # [ 4.390786] (udev-worker)[94]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory lb # [ 4.248159] systemd[1]: Stopped target Timer Units. lb # [ 4.250552] systemd[1]: dbus.socket: Deactivated successfully. client # [ 4.396961] (udev-worker)[94]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.256372] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.258411] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.260281] systemd[1]: Stopped Find NixOS closure. lb # [ 4.261492] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.268673] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.270742] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.274356] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.276809] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.279189] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 4.429619] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.285533] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.286642] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.287651] systemd[1]: Stopped Create System Files and Directories. client # [ 4.432272] systemd[1]: Stopped Virtual Console Setup. client # [ 4.436981] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.293583] systemd[1]: Stopped target Local File Systems. client # [ 4.438916] systemd[1]: Starting Virtual Console Setup... lb # [ 4.296337] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.297356] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.300495] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.302129] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.303299] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.304477] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.305240] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 4.306178] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 4.451506] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.307094] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.453196] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.312272] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.374408] (udev-worker)[97]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.460816] systemd[1]: Starting Virtual Console Setup... lb # [ 4.316193] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.317480] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.318637] systemd[1]: systemd-udevd.service: Consumed 1.126s CPU time over 2.318s wall clock time, 18.3M memory peak. worker1 # [ 4.380253] (udev-worker)[97]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.324179] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 4.325247] systemd[1]: Closed udev Control Socket. lb # [ 4.325968] systemd[1]: Starting Cleanup udev Database... lb # [ 4.326731] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 4.332360] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 4.395481] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.333396] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.334516] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.335489] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 4.399924] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.340538] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 4.404232] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.405039] systemd[1]: Starting Virtual Console Setup... client # [ 4.489383] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 4.492997] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.411087] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.413385] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.355475] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 4.418931] systemd[1]: Starting Virtual Console Setup... lb # [ 4.358480] systemd[1]: Finished Cleanup udev Database. lb # [ 4.360525] systemd[1]: Reached target Switch Root. lb # [ 4.364525] systemd[1]: Starting NixOS Activation... worker2 # [ 4.370522] (udev-worker)[94]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.376143] (udev-worker)[94]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.445245] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.448497] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.416786] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.418054] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.418963] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.421129] systemd[1]: Starting Virtual Console Setup... lb # [ 4.437337] initrd-nixos-activation-start[168]: booting system configuration /nix/store/hw6sjiya3fqcxqx5sjqhqri1ayky0v1l-nixos-system-lb-test worker2 # [ 4.449280] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.452876] systemd[1]: Finished Virtual Console Setup. lb # [ 4.469311] initrd-nixos-activation-start[168]: running activation script... lb # [ 4.698562] initrd-nixos-activation-start[191]: setting up /etc... client # [ 4.859781] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.810099] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.844777] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.928094] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.846301] systemd[1]: Reached target Initrd File Systems. client # [ 4.929731] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.848885] systemd[1]: Starting Find NixOS closure... client # [ 4.933687] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.855784] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.941716] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.815606] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 4.818619] systemd[1]: Finished NixOS Activation. lb # [ 4.819793] systemd[1]: Starting Switch Root... worker1 # [ 4.885103] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.887546] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 4.836658] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.899388] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.902281] systemd[1]: Reached target Initrd Default Target. lb # [ 4.843672] systemd[1]: Switching root. worker1 # [ 4.908277] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.997009] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 5.003329] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.931378] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.014909] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.933082] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.017712] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.936812] systemd[1]: Stopped target Initrd Default Target. client # [ 5.019424] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.937879] systemd[1]: Stopped target Basic System. worker1 # [ 4.940191] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.941082] systemd[1]: Stopped target Path Units. worker1 # [ 4.942648] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.948946] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.950971] systemd[1]: Stopped target Slice Units. worker1 # [ 4.958537] systemd[1]: Stopped target Socket Units. worker1 # [ 4.959333] systemd[1]: Stopped target System Initialization. worker1 # [ 4.960401] systemd[1]: Stopped target Swaps. worker1 # [ 4.961043] systemd[1]: Stopped target Timer Units. worker1 # [ 4.961722] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.962515] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.963317] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 4.965478] systemd[1]: Stopped Find NixOS closure. worker2 # [ 4.903722] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.966216] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 4.967041] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.049377] systemd[1]: Stopped target Initrd Default Target. client # [ 5.051658] systemd[1]: Stopped target Basic System. worker2 # [ 4.907692] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.910303] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.967981] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 4.974145] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.056620] systemd[1]: Stopped target Initrd Root Device. client # [ 5.057763] systemd[1]: Stopped target Path Units. client # [ 5.058745] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 5.061652] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 4.918125] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.983766] systemd[1]: Stopped Load Kernel Modules. client # [ 5.068153] systemd[1]: Stopped target Slice Units. worker1 # [ 4.985622] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.069136] systemd[1]: Stopped target Socket Units. worker1 # [ 4.986764] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.070037] systemd[1]: Stopped target System Initialization. worker1 # [ 4.987851] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.071072] systemd[1]: Stopped target Swaps. worker1 # [ 4.989786] systemd[1]: Stopped Create System Files and Directories. client # [ 5.072449] systemd[1]: Stopped target Timer Units. worker1 # [ 4.990681] systemd[1]: Stopped target Local File Systems. worker1 # [ 4.991455] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 4.992702] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.075436] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.993694] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 4.994457] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 4.995466] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.080526] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.081524] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.082732] systemd[1]: Stopped Find NixOS closure. client # [ 5.083508] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.004192] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.004993] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.006179] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.008566] systemd[1]: systemd-udevd.service: Consumed 1.412s CPU time over 3.045s wall clock time, 22.4M memory peak. client # [ 5.095098] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 5.016188] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.017185] systemd[1]: Closed udev Control Socket. worker1 # [ 5.017877] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.018631] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.101636] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 4.957733] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.960360] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 5.104889] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.106938] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.024425] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.025337] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.026433] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.109706] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.027395] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.032502] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 4.971865] systemd[1]: Finished Find NixOS closure. client # [ 5.115538] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.033368] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.034318] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.117011] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 4.974358] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.975804] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 5.120206] systemd[1]: Stopped Create System Files and Directories. client # [ 5.121188] systemd[1]: Stopped target Local File Systems. client # [ 5.122866] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.126928] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.128162] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.129129] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 5.130161] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.131318] systemd[1]: Stopped Virtual Console Setup. client # [ 5.132189] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.133116] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.134016] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.134947] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.052485] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.135929] systemd[1]: systemd-udevd.service: Consumed 1.403s CPU time over 3.152s wall clock time, 22.3M memory peak. worker1 # [ 5.054685] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.056588] systemd[1]: Reached target Switch Root. worker1 # [ 5.060527] systemd[1]: Starting NixOS Activation... client # [ 5.144493] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 5.145604] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.148422] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.006745] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.152304] systemd[1]: Closed udev Control Socket. client # [ 5.153107] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.008627] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.153886] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.156209] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.012929] systemd[1]: Stopped target Initrd Default Target. lb # [ 5.024595] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.016221] systemd[1]: Stopped target Basic System. client # [ 5.160336] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.017164] systemd[1]: Stopped target Initrd Root Device. client # [ 5.161511] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.018468] systemd[1]: Stopped target Path Units. client # [ 5.162528] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.164215] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.024207] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 5.025466] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.026662] systemd[1]: Stopped target Slice Units. worker2 # [ 5.027411] systemd[1]: Stopped target Socket Units. worker2 # [ 5.029283] systemd[1]: Stopped target System Initialization. worker2 # [ 5.032186] systemd[1]: Stopped target Swaps. worker2 # [ 5.033878] systemd[1]: Stopped target Timer Units. worker2 # [ 5.035434] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 5.038118] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 5.044414] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.189785] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.047648] systemd[1]: Stopped Find NixOS closure. client # [ 5.192988] systemd[1]: Finished Cleanup udev Database. client # [ 5.194620] systemd[1]: Reached target Switch Root. worker2 # [ 5.051560] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.196732] systemd[1]: Starting NixOS Activation... worker2 # [ 5.053937] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.055357] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.057248] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.058262] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 5.058991] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.064189] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.066436] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.071640] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.073173] systemd[1]: Stopped target Local File Systems. worker2 # [ 5.073952] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.074895] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.075875] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 5.076934] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.077963] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.078972] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.079693] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.080778] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.141081] initrd-nixos-activation-start[177]: booting system configuration /nix/store/hmsqrrazdjx0ansfssikbgws6gj8896v-nixos-system-worker1-test worker2 # [ 5.081598] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.082515] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.083495] systemd[1]: systemd-udevd.service: Consumed 1.400s CPU time over 3.085s wall clock time, 22.4M memory peak. worker2 # [ 5.089336] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.092215] systemd[1]: Closed udev Control Socket. worker2 # [ 5.096215] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.097152] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.098211] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.100142] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.104178] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.105196] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.106160] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.176800] initrd-nixos-activation-start[177]: running activation script... worker2 # [ 5.133311] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.136548] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.137408] systemd[1]: Reached target Switch Root. worker2 # [ 5.138817] systemd[1]: Starting NixOS Activation... client # [ 5.290986] initrd-nixos-activation-start[176]: booting system configuration /nix/store/ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test client # [ 5.328259] initrd-nixos-activation-start[176]: running activation script... worker2 # [ 5.227326] initrd-nixos-activation-start[176]: booting system configuration /nix/store/bw3mqal5y2l6sysyr6dir665462635vh-nixos-system-worker2-test worker2 # [ 5.259757] initrd-nixos-activation-start[176]: running activation script... worker1 # [ 5.436370] initrd-nixos-activation-start[200]: setting up /etc... client # [ 5.602647] initrd-nixos-activation-start[199]: setting up /etc... worker1 # [ 5.575142] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.514517] initrd-nixos-activation-start[199]: setting up /etc... worker1 # [ 5.578165] systemd[1]: Finished NixOS Activation. worker1 # [ 5.580146] systemd[1]: Starting Switch Root... worker1 # [ 5.603527] systemd[1]: Switching root. lb # [ 5.552807] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 5.565902] systemd[1]: Detected virtualization qemu. lb # [ 5.569166] systemd[1]: Detected architecture arm64. lb # [ 5.573197] systemd[1]: Detected first boot. lb # [ 5.577907] systemd[1]: Initializing machine ID from random generator. client # [ 5.722729] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.725654] systemd[1]: Finished NixOS Activation. client # [ 5.726860] systemd[1]: Starting Switch Root... client # [ 5.747472] systemd[1]: Switching root. worker2 # [ 5.635287] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.638131] systemd[1]: Finished NixOS Activation. worker2 # [ 5.640164] systemd[1]: Starting Switch Root... worker2 # [ 5.663011] systemd[1]: Switching root. worker1 # [ 5.786389] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.937196] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.846583] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.897994] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.110163] systemd[1]: Applying preset policy. worker1 # [ 6.294846] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.307488] systemd[1]: Detected virtualization qemu. worker1 # [ 6.310851] systemd[1]: Detected architecture arm64. worker1 # [ 6.314563] systemd[1]: Detected first boot. worker1 # [ 6.320118] systemd[1]: Initializing machine ID from random generator. client # [ 6.466826] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.479597] systemd[1]: Detected virtualization qemu. client # [ 6.482663] systemd[1]: Detected architecture arm64. client # [ 6.486617] systemd[1]: Detected first boot. client # [ 6.492481] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.368759] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.381406] systemd[1]: Detected virtualization qemu. worker2 # [ 6.384471] systemd[1]: Detected architecture arm64. worker2 # [ 6.388408] systemd[1]: Detected first boot. lb # [ 6.388302] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 6.394060] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.644877] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.621161] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.622473] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.625817] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.629108] systemd[1]: Created slice Slice /system/getty. lb # [ 6.631255] systemd[1]: Created slice User and Session Slice. lb # [ 6.632534] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 6.634489] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 6.636346] systemd[1]: Expecting device /dev/hvc0... lb # [ 6.638854] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 6.639900] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 6.641575] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.643209] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 6.645697] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 6.646860] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 6.647174] systemd[1]: Reached target Path Units. lb # [ 6.647425] systemd[1]: Reached target Remote File Systems. lb # [ 6.647671] systemd[1]: Reached target Slice Units. lb # [ 6.647918] systemd[1]: Reached target Swaps. lb # [ 6.652968] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 6.656039] systemd[1]: Listening on Process Core Dump Socket. lb # [ 6.658251] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 6.660716] systemd[1]: Listening on Factory Reset Management. lb # [ 6.661949] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.666208] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.668546] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.671430] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.673029] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 6.818332] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.674666] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.677098] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 6.683393] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.683778] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 6.685104] systemd[1]: Listening on udev Control Socket. lb # [ 6.686627] systemd[1]: Listening on udev Varlink Socket. lb # [ 6.689948] systemd[1]: Mounting Huge Pages File System... lb # [ 6.695242] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 6.711864] systemd[1]: Mounting Kernel Debug File System... lb # [ 6.716064] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.727900] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.729971] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 6.745179] systemd[1]: Mounting Kernel Configuration File System... lb # [ 6.746430] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 6.751308] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 6.753239] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.782356] systemd[1]: Mounting FUSE Control File System... lb # [ 6.783645] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 6.864040] systemd[1]: Applying preset policy. lb # [ 6.820435] systemd[1]: Starting Journal Service... lb # [ 6.834977] systemd[1]: Starting Load Kernel Modules... lb # [ 6.846570] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 6.854280] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 6.857596] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.878429] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.028124] systemd[1]: Applying preset policy. lb # [ 6.887691] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 6.893472] systemd[1]: Applying preset policy. lb # [ 6.902452] systemd-journald[262]: Collecting audit messages is enabled. lb # [ 6.909444] systemd[1]: Mounted Huge Pages File System. lb # [ 6.912491] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 6.913784] systemd[1]: Mounted Kernel Debug File System. lb # [ 6.907101] systemd[1]: Queued start job for default target Multi-User System. lb # [ 6.926465] systemd[1]: Started Journal Service. lb # [ 6.915164] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 6.919086] systemd[1]: Mounted Kernel Trace File System. lb # [ 6.928411] systemd-modules-load[263]: Module 'atkbd' is built in lb # [ 6.929530] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 6.930553] systemd-modules-load[263]: Module 'loop' is built in lb # [ 6.942787] systemd[1]: Finished Load Kernel Modules. lb # [ 6.951117] systemd[1]: Mounted Kernel Configuration File System. lb # [ 6.962260] systemd[1]: Starting Firewall... lb # [ 6.965920] systemd[1]: Starting Apply Kernel Variables... lb # [ 6.979302] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 6.994879] systemd-oomd[265]: No swap; memory pressure usage will be degraded lb # [ 7.003703] systemd[1]: Mounted FUSE Control File System. lb # [ 7.023004] EXT4-fs (vda): re-mounted 14220a4a-ec47-41b7-9745-bca8042d2bff. lb # [ 7.024177] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.101640] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.033652] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.039647] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 7.051774] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 7.054542] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.064322] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 7.065170] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.139016] systemd[1]: Populated /etc with preset unit settings. client # [ 7.296220] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.155780] systemd-journald[262]: Received client request to flush runtime journal. lb # [ 7.208925] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.209901] systemd[1]: Reached target First Boot Complete. lb # [ 7.210698] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.216587] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 7.323123] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.324414] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.329023] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.332482] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.334591] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.335840] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.337664] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.340148] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.341937] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.343802] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.344909] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.346439] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.348692] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.350684] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.351856] systemd[1]: Reached target Path Units. worker1 # [ 7.353427] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.355684] systemd[1]: Reached target Slice Units. worker1 # [ 7.356644] systemd[1]: Reached target Swaps. worker1 # [ 7.359417] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.362486] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.364705] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.367073] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.368272] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.372226] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.374371] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.377915] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.379470] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.381483] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.383199] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.389211] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.390638] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.392191] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.395758] systemd[1]: Mounting Huge Pages File System... lb # [ 7.324847] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 7.403753] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.333240] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 7.410149] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.365325] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.423766] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.367105] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.371518] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.376079] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.378369] systemd[1]: Created slice User and Session Slice. client # [ 7.521835] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.437425] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.379660] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.437816] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.523148] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.381587] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.384059] systemd[1]: Expecting device /dev/hvc0... client # [ 7.527426] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.386055] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.387125] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.531521] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.388719] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.390952] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.533629] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.392034] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.534894] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.393725] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.452612] systemd[1]: Mounting Kernel Configuration File System... client # [ 7.537165] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.396138] systemd[1]: Reached target Path Units. worker1 # [ 7.452985] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.539079] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.453255] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.397192] systemd[1]: Reached target Remote File Systems. client # [ 7.540891] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.453518] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.399436] systemd[1]: Reached target Slice Units. client # [ 7.542554] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.400425] systemd[1]: Reached target Swaps. client # [ 7.544394] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.403353] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.546016] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.547743] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.406362] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.548854] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.550321] systemd[1]: Reached target Path Units. worker2 # [ 7.408672] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.551623] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.411015] systemd[1]: Listening on Factory Reset Management. client # [ 7.553466] systemd[1]: Reached target Slice Units. worker2 # [ 7.412212] systemd[1]: Listening on Hostname Service Socket. client # [ 7.555062] systemd[1]: Reached target Swaps. client # [ 7.558087] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.416442] systemd[1]: Starting Journal Log Access Socket... client # [ 7.561007] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.418536] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.476981] systemd[1]: Mounting FUSE Control File System... client # [ 7.563176] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.421490] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.477383] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.565611] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.423155] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.566835] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.424653] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.570856] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.427763] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.573009] systemd[1]: Listening on Journal Audit Socket. client # [ 7.576406] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.433737] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.435092] systemd[1]: Listening on udev Control Socket. client # [ 7.577989] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.436397] systemd[1]: Listening on udev Varlink Socket. client # [ 7.579347] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.439728] systemd[1]: Mounting Huge Pages File System... client # [ 7.581637] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.588531] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 7.589898] systemd[1]: Listening on udev Control Socket. client # [ 7.591323] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.448705] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.594588] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.458109] systemd[1]: Mounting Kernel Debug File System... client # [ 7.600734] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.517898] systemd[1]: Starting Journal Service... worker2 # [ 7.468900] systemd[1]: Mounting Kernel Trace File System... client # [ 7.612077] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.480796] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.539974] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.482023] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.626095] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.550411] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.492889] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.495364] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.640275] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.642371] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.560077] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.500903] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.561683] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.501262] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.568817] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.656104] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.573788] systemd[1]: Listening on Journal Log Access Socket. client # [ 7.659661] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.578329] systemd[1]: Mounted Huge Pages File System. worker1 # [ 7.580859] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.524805] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.525198] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.586333] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.586926] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.587427] systemd[1]: Mounted Kernel Configuration File System. client # [ 7.673079] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.587829] systemd[1]: Mounted FUSE Control File System. client # [ 7.675985] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.600828] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.691198] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.607685] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 7.550822] systemd[1]: Starting Journal Service... client # [ 7.692429] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.714280] systemd[1]: Starting Journal Service... worker2 # [ 7.572338] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.590598] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.736112] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.596562] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.598408] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.604410] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.590071] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 7.607858] systemd[1]: Listening on Journal Log Access Socket. lb # [ 7.591124] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 7.612946] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.614383] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.597682] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 7.616534] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.618326] systemd[1]: Mounted Kernel Trace File System. client # [ 7.763453] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.621136] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.627241] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.683474] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.694779] systemd[1]: Starting Firewall... lb # [ 7.623255] systemd[1]: Mounting /run/wrappers... client # [ 7.783061] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.783447] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.706720] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.649321] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.709956] systemd-journald[270]: Collecting audit messages is enabled. worker2 # [ 7.654877] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 7.717648] EXT4-fs (vda): re-mounted 85753b96-c7ba-477e-964a-82432a2541e5. client # [ 7.802819] systemd-journald[269]: Collecting audit messages is enabled. client # [ 7.809082] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.727743] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.730437] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.820398] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.732729] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.808896] systemd[1]: Queued start job for default target Multi-User System. client # [ 7.810400] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 7.811842] systemd-modules-load[270]: Module 'atkbd' is built in worker1 # [ 7.746837] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.832085] systemd[1]: Started Journal Service. client # [ 7.819218] systemd-modules-load[270]: Module 'loop' is built in worker1 # [ 7.750135] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.822850] systemd[1]: Mounted Huge Pages File System. client # [ 7.823814] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.828754] systemd[1]: Mounted Kernel Debug File System. client # [ 7.829693] systemd[1]: Mounted Kernel Trace File System. client # [ 7.836297] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.839751] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.772712] systemd[1]: Started Journal Service. worker1 # [ 7.766717] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.722674] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.771222] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.710289] systemd[1]: Mounted /run/wrappers. lb # [ 7.711113] systemd[1]: Reached target Local File Systems. worker2 # [ 7.732469] systemd[1]: Starting Firewall... client # [ 7.875918] EXT4-fs (vda): re-mounted 029cd0ef-f3c7-49c5-af97-28279bde576d. lb # [ 7.718409] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 7.784676] systemd-modules-load[271]: Module 'atkbd' is built in worker2 # [ 7.742278] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.726537] systemd[1]: Starting register-nix-paths.service... lb # [ 7.731669] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 7.877569] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.797759] systemd-modules-load[271]: Module 'loop' is built in client # [ 7.881283] systemd[1]: Mounted Kernel Configuration File System. lb # [ 7.736210] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 7.758911] systemd-journald[269]: Collecting audit messages is enabled. lb # [ 7.744074] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 7.889827] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.764704] EXT4-fs (vda): re-mounted e2934ba0-53e2-4b9d-9be0-9be4a9393436. worker1 # [ 7.811212] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.897151] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.898173] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.904806] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.912255] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.781010] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.768116] systemd[1]: Starting Create System Files and Directories... worker2 # [ 7.781803] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.782214] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.917061] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.920162] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.796564] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.924593] systemd[1]: Mounted FUSE Control File System. worker2 # [ 7.798009] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.866314] systemd-oomd[272]: No swap; memory pressure usage will be degraded client # [ 7.948950] systemd-oomd[272]: No swap; memory pressure usage will be degraded worker2 # [ 7.824324] systemd[1]: Started Journal Service. worker2 # [ 7.820669] systemd[1]: Queued start job for default target Multi-User System. client # [ 7.966854] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.824246] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 7.891939] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.838398] systemd-modules-load[270]: Module 'atkbd' is built in worker1 # [ 7.906373] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 7.850858] systemd-modules-load[270]: Module 'loop' is built in worker1 # [ 7.913926] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 7.920648] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 7.860118] systemd[1]: Starting Flush Journal to Persistent Storage... worker1 # [ 7.921529] systemd[1]: Reached target First Boot Complete. client # [ 8.021293] systemd-journald[269]: Received client request to flush runtime journal. worker1 # [ 7.944623] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 7.987854] systemd-journald[270]: Received client request to flush runtime journal. client # [ 8.064840] systemd[1]: Finished Load/Save OS Random Seed. client # [ 8.065800] systemd[1]: Reached target First Boot Complete. client # [ 8.072505] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 7.929836] systemd-oomd[271]: No swap; memory pressure usage will be degraded lb # [ 7.935543] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 7.947840] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 7.950742] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 7.951713] systemd[1]: Reached target First Boot Complete. lb # [ 7.955514] systemd-udevd[302]: Using default interface naming scheme 'v261'. worker2 # [ 7.966586] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.967506] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 8.116307] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.989292] systemd-journald[269]: Received client request to flush runtime journal. worker1 # [ 8.061095] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.090473] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.096416] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.100259] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.057255] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.060676] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.069514] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 8.073384] systemd[1]: Finished Create System Files and Directories. lb # [ 8.090472] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 8.098568] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.153089] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.157835] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.219341] systemd-udevd[311]: Using default interface naming scheme 'v261'. worker2 # [ 8.168440] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.372965] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.239915] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.385085] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.313579] systemd[1]: Mounting /run/wrappers... lb # [ 8.263062] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 8.295370] systemd-udevd[311]: Using default interface naming scheme 'v261'. lb # [ 8.318449] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.382820] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.328863] systemd[1]: Starting Update is Completed... worker1 # [ 8.397930] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.406629] systemd[1]: Reached target Local File Systems. worker1 # [ 8.410297] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.352562] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.415712] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.424627] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.440722] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.453764] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.457369] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.440748] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.444252] systemd[1]: Reached target Local File Systems. worker2 # [ 8.448179] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 8.448469] systemd[1]: Finished Update is Completed. worker2 # [ 8.459426] systemd[1]: Starting register-nix-paths.service... worker2 # [ 8.469565] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 8.470637] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.488207] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.492410] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.566607] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.572899] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.534799] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.746981] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 8.604333] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.611601] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.755186] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.758103] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.763004] systemd[1]: Mounting /run/wrappers... client # [ 8.765164] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.688157] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.696907] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 8.706205] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.812980] systemd[1]: Mounted /run/wrappers. client # [ 8.815140] systemd[1]: Reached target Local File Systems. client # [ 8.818597] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.822201] systemd[1]: Starting register-nix-paths.service... lb # [ 8.677398] systemd[1]: Finished Coldplug All udev Devices. client # [ 8.829481] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.830841] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.837743] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.860807] systemd-udevd[300]: Using default interface naming scheme 'v261'. client # [ 8.871417] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.745414] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.753500] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 8.769154] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 8.842213] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.956789] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.969846] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.920624] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.924887] systemd[1]: Starting Update is Completed... worker2 # [ 8.892833] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.043058] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 9.050927] systemd[1]: Finished Create System Files and Directories. client # [ 9.059040] systemd[1]: Starting Rebuild Journal Catalog... client # [ 9.068187] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 9.011477] systemd[1]: Finished Update is Completed. worker2 # [ 8.975742] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 8.979721] systemd[1]: Starting Update is Completed... client # [ 9.164450] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.214066] systemd[1]: Finished Rebuild Journal Catalog. client # [ 9.222069] systemd[1]: Starting Update is Completed... worker2 # [ 9.079022] systemd[1]: Finished Update is Completed. lb # [ 9.116204] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.121478] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.293244] systemd[1]: Finished Update is Completed. lb # [ 9.258114] systemd[1]: Finished register-nix-paths.service. lb # [ 9.259145] systemd[1]: Reached target System Initialization. lb # [ 9.266136] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.267300] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.272270] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 9.273269] systemd[1]: Reached target Timer Units. lb # [ 9.274001] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 9.279572] systemd[1]: Listening on niks3 server socket. lb # [ 9.282093] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.282901] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.301448] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 9.291524] systemd[1]: Reached target Socket Units. lb # [ 9.294804] systemd[1]: Reached target Basic System. lb # [ 9.295547] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 9.299260] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.304413] systemd[1]: Starting Post-Boot Actions... lb # [ 9.308463] systemd[1]: Started Reset console on configuration changes. lb # [ 9.328311] systemd[1]: Starting resolvconf update... worker1 # [ 9.414169] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.416736] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.384428] systemd[1]: Started rustfs.service. lb # [ 9.431290] systemd[1]: Starting rustfs-bucket.service... client # [ 9.602258] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 9.463518] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 9.469480] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.470775] systemd[1]: Starting D-Bus System Message Bus... lb # [ 9.496966] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 9.647960] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.505810] nsncd[446]: Sep 22 11:25:08.098 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.551488] systemd[1]: Finished Post-Boot Actions. lb # [ 9.612112] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.613190] systemd[1]: Reached target User and Group Name Lookups. client # [ 9.770979] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.774546] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.652377] systemd[1]: Starting User Login Management... client # [ 9.807444] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 9.753837] systemd[1]: Finished Firewall. client # [ 9.858393] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 9.788104] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 9.804109] systemd[1]: Finished Firewall. client # [ 10.036660] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.045793] (udev-worker)[357]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 9.906275] dbus-broker-launch[455]: Looking up NSS user entry for 'systemd-timesync'... client # [ 10.048004] (udev-worker)[357]: Network interface NamePolicy= disabled on kernel command line. lb # [ 9.917737] dbus-broker-launch[455]: NSS returned no entry for 'systemd-timesync' lb # [ 9.918924] dbus-broker-launch[455]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 9.971128] systemd-logind[484]: New seat seat0. lb # [ 9.990380] systemd[1]: Started User Login Management. lb # [ 10.000136] systemd[1]: Starting linger-users.service... lb # [ 10.008346] systemd[1]: Started D-Bus System Message Bus. lb # [ 10.019119] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.024283] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.025248] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 10.026132] systemd[1]: Stopping User and Group Name Lookups... lb # [ 10.026974] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 10.049228] systemd[1]: nscd.service: Deactivated successfully. lb # [ 10.053707] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 10.125318] systemd[1]: Finished Coldplug All udev Devices. lb # [ 10.097625] dbus-broker-launch[455]: Ready lb # [ 10.112517] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.205847] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 10.232959] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 10.331054] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.189348] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.335658] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 10.192565] systemd[1]: linger-users.service: Deactivated successfully. client # [ 10.340986] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 10.341885] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 10.343261] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 10.201383] systemd[1]: Finished linger-users.service. worker1 # [ 10.271018] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.273135] systemd[1]: Reached target System Initialization. client # [ 10.356923] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 10.275024] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.358597] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 10.359867] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 10.280856] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.367630] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.286595] systemd[1]: Reached target Timer Units. worker1 # [ 10.295594] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.244831] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.309365] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.251187] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.257302] nsncd[548]: Sep 22 11:25:08.851 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.321386] systemd[1]: Listening on Nix worker-protocol over gRPC. lb # [ 10.259320] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.325677] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.268182] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 10.335795] systemd[1]: Reached target Socket Units. worker1 # [ 10.340253] systemd[1]: Reached target Basic System. worker1 # [ 10.346388] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 10.444776] mousedev: PS/2 mouse device common for all mice worker2 # [ 10.287592] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 10.356597] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.369217] systemd[1]: Starting Post-Boot Actions... lb # [ 10.312218] systemd[1]: Finished resolvconf update. worker1 # [ 10.380453] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.389027] systemd[1]: Starting resolvconf update... lb # [ 10.329659] systemd[1]: Starting DHCP Client... worker1 # [ 10.396372] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.342381] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.347685] systemd[1]: Reached target System Initialization. worker2 # [ 10.350485] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 10.353027] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.355051] systemd[1]: Reached target Timer Units. worker2 # [ 10.357330] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.361994] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.366901] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 10.439250] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.380508] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.393097] systemd[1]: Reached target Socket Units. worker2 # [ 10.405300] systemd[1]: Reached target Basic System. worker1 # [ 10.469118] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.409391] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.416871] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.478564] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.419650] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.484706] nsncd[487]: Sep 22 11:25:09.081 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.426409] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.436460] systemd[1]: Starting resolvconf update... worker1 # [ 10.500442] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.506201] systemd[1]: Starting User Login Management... worker2 # [ 10.445871] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.537451] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.545297] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.552447] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.558789] dbus-broker-launch[491]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.571732] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.526418] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.549584] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.558556] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.630879] dbus-broker-launch[491]: Ready worker2 # [ 10.569522] nsncd[487]: Sep 22 11:25:09.166 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.580932] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.589651] systemd[1]: Starting User Login Management... worker2 # [ 10.617238] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.627567] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.635448] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.696148] systemd-logind[508]: New seat seat0. worker1 # [ 10.703700] systemd[1]: Started User Login Management. worker2 # [ 10.641178] dbus-broker-launch[491]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.713214] systemd[1]: Starting linger-users.service... lb # [ 10.653355] dhcpcd[595]: dhcpcd-10.3.2 starting worker2 # [ 10.657271] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.722947] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.734285] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.670916] dhcpcd[613]: dev: loaded udev lb # [ 10.675845] dhcpcd[613]: no valid interfaces found worker1 # [ 10.738796] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 10.678341] dhcpcd[613]: no valid interfaces found lb # [ 10.679073] dhcpcd[613]: libudev: received NULL device lb # [ 10.679831] dhcpcd[613]: libudev: received NULL device worker1 # [ 10.746534] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.749713] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.753859] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 10.759982] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 10.766018] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.708001] dbus-broker-launch[491]: Ready lb # [ 10.718182] systemd[1]: Starting Virtual Console Setup... worker1 # [ 10.820384] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 10.758807] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 10.824496] systemd[1]: Finished linger-users.service. worker2 # [ 10.766895] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 10.918555] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 10.775194] systemd-logind[503]: New seat seat0. worker1 # [ 10.836256] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 10.921359] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 10.778105] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 10.924277] systemd[1]: Reached target System Initialization. worker1 # [ 10.844313] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.783470] systemd[1]: Started User Login Management. lb # [ 10.783401] systemd[1]: Stopped Virtual Console Setup. client # [ 10.928990] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.930734] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.848338] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.788159] systemd[1]: Starting linger-users.service... client # [ 10.935469] systemd[1]: Reached target Timer Units. worker1 # [ 10.851624] nsncd[554]: Sep 22 11:25:09.457 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 10.937067] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 10.938032] systemd[1]: Listening on Nix Daemon Socket. client # [ 10.938835] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.860076] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.946297] systemd[1]: Reached target Socket Units. client # [ 10.948242] systemd[1]: Reached target Basic System. lb # [ 10.804825] systemd[1]: Starting Virtual Console Setup... worker2 # [ 10.808207] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 10.952255] systemd[1]: Started backdoor.service. client # [ 10.953677] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.811523] systemd[1]: Stopping Host and Network Name Lookups... client # [ 10.957668] systemd[1]: Starting nftables firewall... worker1 # [ 10.876585] systemd[1]: Finished resolvconf update. worker2 # [ 10.816885] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.878563] systemd[1]: Reached target Preparation for Network. client # [ 10.961465] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.882356] systemd[1]: Starting DHCP Client... worker2 # [ 10.823526] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.888289] systemd[1]: Starting Extra networking commands.... worker2 # [ 10.833027] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.841420] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.845313] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.847919] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.999329] systemd[1]: Starting Post-Boot Actions... lb # [ 10.878690] systemd[1]: Finished Firewall. lb # [ 10.883106] systemd[1]: Reached target Preparation for Network. client # [ 11.030396] systemd[1]: Started Reset console on configuration changes. lb # [ 10.891624] systemd[1]: Starting Extra networking commands.... worker1 # [ 10.957313] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.898812] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 10.961153] systemd[1]: Started backdoor.service. worker2 # [ 10.904757] systemd[1]: Finished linger-users.service. worker2 # [ 10.912308] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.919168] systemd[1]: Started backdoor.service. client # [ 11.070863] systemd[1]: Starting resolvconf update... worker2 # [ 10.945206] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.948186] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.953212] nsncd[557]: Sep 22 11:25:09.564 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.962530] systemd[1]: Reached target User and Group Name Lookups. client # connecting to host... worker2 # [ 10.975707] systemd[1]: Finished resolvconf update. worker2 # [ 10.983494] systemd[1]: Reached target Preparation for Network. worker2 # [ 10.993829] systemd[1]: Starting DHCP Client... worker2 # [ 11.000902] systemd[1]: Starting Extra networking commands.... client: Guest shell says: b'Spawning backdoor root shell...\n' worker1 # connecting to host... client # [ 11.170256] systemd[1]: Starting D-Bus System Message Bus... worker2 # connecting to host... client # [ 11.195146] nsncd[452]: Sep 22 11:25:09.811 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.205532] systemd[1]: Finished Post-Boot Actions. client # [ 11.206352] systemd[1]: Started Name Service Cache Daemon (nsncd). client: connected to guest root shell client: (connecting took 11.48 seconds) client: (finished: waiting for the VM to finish booting, in 11.48 seconds) client # [ 11.257042] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 11.179231] (udev-worker)[351]: Network interface NamePolicy= disabled on kernel command line. client # [ 11.264700] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.265695] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.185569] dhcpcd[590]: dhcpcd-10.3.2 starting worker1 # [ 11.197983] dhcpcd[632]: dev: loaded udev worker1 # [ 11.201854] (udev-worker)[348]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 11.289223] systemd[1]: Starting User Login Management... worker1 # [ 11.209735] (udev-worker)[348]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.246774] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.199420] (udev-worker)[353]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.206232] (udev-worker)[343]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 11.217770] (udev-worker)[343]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.306453] systemd[1]: Finished Extra networking commands.. worker1 # [ 11.307363] systemd[1]: Reached target Network. worker1 # [ 11.316199] systemd[1]: Starting Permit User Sessions... worker2 # [ 11.263056] dhcpcd[594]: dhcpcd-10.3.2 starting client # [ 11.414436] dbus-broker-launch[464]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 11.273914] dhcpcd[637]: dev: loaded udev lb # [ 11.276459] systemd[1]: Finished Extra networking commands.. lb # [ 11.282775] systemd[1]: Reached target Network. worker1 # [ 11.360854] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 11.433414] dbus-broker-launch[464]: NSS returned no entry for 'systemd-timesync' client # [ 11.434554] dbus-broker-launch[464]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 11.295862] systemd[1]: Starting PostgreSQL Server... worker1 # [ 11.362762] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.320894] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.305201] systemd[1]: Starting Permit User Sessions... worker1 # [ 11.368716] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.405894] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.407607] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 11.409740] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.410110] cfg80211: failed to load regulatory.db client # [ 11.494392] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 11.419294] systemd[1]: Finished Permit User Sessions. worker2 # [ 11.360448] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.361447] systemd[1]: Reached target Network. worker1 # [ 11.425154] systemd[1]: Started Getty on tty1. worker1 # [ 11.425879] systemd[1]: Reached target Login Prompts. worker2 # [ 11.378497] systemd[1]: Starting Permit User Sessions... worker2 # [ 11.429565] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.415883] systemd[1]: Finished Permit User Sessions. client # [ 11.560844] dbus-broker-launch[464]: Ready worker1 # [ 11.502409] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.430063] systemd[1]: Started Getty on tty1. client # [ 11.574988] systemd-logind[486]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.432513] systemd[1]: Reached target Login Prompts. worker2 # [ 11.436495] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 11.587923] systemd-logind[486]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 11.445005] systemd[1]: Starting Address configuration of eth1... client # [ 11.592330] systemd-logind[486]: New seat seat0. client # [ 11.593100] systemd[1]: Started User Login Management. worker1 # [ 11.524439] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.514274] network-addresses-eth1-start[652]: adding address 192.168.1.3/24... done worker1 # [ 11.519878] dhcpcd[632]: eth0: waiting for carrier client # [ 11.604409] systemd[1]: Starting linger-users.service... worker1 # [ 11.522400] dhcpcd[632]: eth0: waiting for carrier worker2 # [ 11.478028] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.478594] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 11.525812] dhcpcd[632]: eth0: carrier acquired worker2 # [ 11.464547] systemd[1]: Finished Permit User Sessions. worker2 # [ 11.486042] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.486380] cfg80211: failed to load regulatory.db worker1 # [ 11.535928] network-addresses-eth1-start[652]: adding address 2001:db8:1::3/64... done worker2 # [ 11.479170] systemd[1]: Started Getty on tty1. worker2 # [ 11.479913] systemd[1]: Reached target Login Prompts. worker1 # [ 11.541408] dhcpcd[632]: DUID 00:01:00:01:32:45:22:16:52:54:00:12:34:56 worker1 # [ 11.542391] dhcpcd[632]: eth0: IAID 00:12:34:56 worker1 # [ 11.543018] dhcpcd[632]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.559139] systemd[1]: Finished Address configuration of eth1. lb # [ 11.497467] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 11.646909] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.647966] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.658372] systemd[1]: Stopped target User and Group Name Lookups. client # [ 11.659366] systemd[1]: Stopping User and Group Name Lookups... client # [ 11.672188] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 11.673125] systemd[1]: nscd.service: Deactivated successfully. client # [ 11.673915] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 11.530646] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 11.540231] systemd[1]: Started backdoor.service. client # [ 11.700765] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 11.571899] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.705017] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.708958] systemd[1]: Finished linger-users.service. worker2 # [ 11.590659] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.578600] dhcpcd[637]: eth0: waiting for carrier worker2 # [ 11.581907] dhcpcd[637]: eth0: waiting for carrier worker2 # [ 11.586302] dhcpcd[637]: eth0: carrier acquired worker2 # [ 11.587412] network-addresses-eth1-start[652]: adding address 192.168.1.4/24... done worker1 # [ 11.662619] dhcpcd[632]: eth0: soliciting a DHCP lease worker2 # [ 11.604248] network-addresses-eth1-start[652]: adding address 2001:db8:1::4/64... done worker1 # [ 11.668660] dhcpcd[632]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 11.607994] dhcpcd[637]: DUID 00:01:00:01:32:45:22:16:52:54:00:12:34:56 worker2 # [ 11.609537] dhcpcd[637]: eth0: IAID 00:12:34:56 worker2 # [ 11.610225] dhcpcd[637]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.676917] dhcpcd[632]: eth0: probing address 10.0.2.15/24 worker2 # [ 11.626431] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.698807] systemd-logind[508]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.783998] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.789735] nsncd[546]: Sep 22 11:25:10.405 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.791778] systemd[1]: Reached target Host and Network Name Lookups. lb # connecting to host... client # [ 11.797917] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.816289] systemd[1]: Finished resolvconf update. client # [ 11.823526] systemd[1]: Starting DHCP Client... lb # [ 11.709458] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.706777] postgresql-pre-start[695]: The files belonging to this database system will be owned by user "postgres". lb # [ 11.710058] postgresql-pre-start[695]: This user must also own the server process. lb # [ 11.718648] systemd-vconsole-setup[620]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 11.727687] postgresql-pre-start[695]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 11.733941] postgresql-pre-start[695]: The default database encoding has accordingly been set to "UTF8". lb # [ 11.735249] postgresql-pre-start[695]: The default text search configuration will be set to "english". lb # [ 11.742790] postgresql-pre-start[695]: Data page checksums are enabled. lb # [ 11.743775] postgresql-pre-start[695]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 11.749596] postgresql-pre-start[695]: creating subdirectories ... ok lb # [ 11.750515] postgresql-pre-start[695]: selecting dynamic shared memory implementation ... posix lb # [ 11.751661] systemd[1]: Finished Virtual Console Setup. lb # [ 11.762651] (udev-worker)[643]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 11.767063] systemd-logind[503]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 11.769532] (udev-worker)[643]: Network interface NamePolicy= disabled on kernel command line. lb # [ 11.771016] (udev-worker)[647]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.895377] mousedev: PS/2 mouse device common for all mice lb # [ 11.867757] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 11.872727] mousedev: PS/2 mouse device common for all mice client # [ 12.031061] dhcpcd[577]: dhcpcd-10.3.2 starting client # [ 12.043422] dhcpcd[597]: dev: loaded udev worker1 # [ 11.972162] systemd-logind[508]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.932132] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.932665] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 11.935729] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.936055] cfg80211: failed to load regulatory.db client # [ 12.101901] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.973953] systemd-logind[503]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.981274] postgresql-pre-start[695]: selecting default "max_connections" ... 100 worker2 # [ 12.035646] dhcpcd[637]: eth0: soliciting a DHCP lease worker2 # [ 12.041043] dhcpcd[637]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 12.048292] dhcpcd[637]: eth0: probing address 10.0.2.15/24 client # [ 12.218260] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 12.107758] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 12.117506] systemd[1]: Starting Address configuration of eth1... client # [ 12.289437] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 12.289965] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 12.293467] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 12.293835] cfg80211: failed to load regulatory.db lb # [ 12.139500] postgresql-pre-start[695]: selecting default "shared_buffers" ... 128MB client # [ 12.334403] systemd[1]: Finished nftables firewall. client # [ 12.337863] systemd[1]: Reached target Preparation for Network. client # [ 12.348410] systemd[1]: Starting Address configuration of eth1... client # [ 12.358214] systemd[1]: Starting Extra networking commands.... lb # [ 12.233705] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 12.222365] dhcpcd[613]: eth0: waiting for carrier lb # [ 12.226459] dhcpcd[613]: eth0: carrier acquired lb # [ 12.255529] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 12.247568] systemd-logind[484]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 12.254576] dhcpcd[613]: DUID 00:01:00:01:32:45:22:16:52:54:00:12:34:56 lb # [ 12.255578] dhcpcd[613]: eth0: IAID 00:12:34:56 client # [ 12.416424] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 12.258908] dhcpcd[613]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 12.405959] dhcpcd[597]: eth0: waiting for carrier client # [ 12.409334] dhcpcd[597]: libudev: received NULL device client # [ 12.410192] dhcpcd[597]: libudev: received NULL device client # [ 12.411032] dhcpcd[597]: eth0: carrier acquired lb # [ 12.278165] network-addresses-eth1-start[718]: adding address 192.168.1.2/24... done client # [ 12.431824] dhcpcd[597]: DUID 00:01:00:01:32:45:22:17:52:54:00:12:34:56 client # [ 12.435064] dhcpcd[597]: eth0: IAID 00:12:34:56 client # [ 12.435781] dhcpcd[597]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.295903] network-addresses-eth1-start[718]: adding address 2001:db8:1::2/64... done lb # [ 12.318899] systemd[1]: Finished Address configuration of eth1. client # [ 12.500476] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 12.516497] network-addresses-eth1-start[618]: adding address 192.168.1.1/24... done client # [ 12.541129] network-addresses-eth1-start[618]: adding address 2001:db8:1::1/64... done lb # [ 12.434406] mousedev: PS/2 mouse device common for all mice client # [ 12.570652] systemd[1]: Finished Address configuration of eth1. client # [ 12.664325] systemd[1]: Finished Extra networking commands.. client # [ 12.668550] systemd[1]: Reached target Network. client # [ 12.676271] systemd[1]: Starting Permit User Sessions... lb # [ 12.531772] systemd-logind[484]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 12.617210] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client: (finished: waiting for unit network-addresses-eth1.service, in 12.95 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } client # [ 12.749855] systemd[1]: Finished Permit User Sessions. client # [ 12.762862] systemd[1]: Started Getty on tty1. client # [ 12.766164] systemd[1]: Reached target Login Prompts. worker2 # [ 12.642428] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.08 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.849305] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb: (finished: waiting for unit network-addresses-eth1.service, in 0.09 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.06 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1 # [ 13.032940] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 13.046580] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.065793] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 13.072489] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.073292] systemd[1]: Starting Virtual Console Setup... worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.09 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2 # [ 13.019339] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.030259] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.054174] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.055328] systemd[1]: Stopped Virtual Console Setup. client # [ 13.200899] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.064618] systemd[1]: Starting Virtual Console Setup... client # [ 13.212636] dhcpcd[597]: eth0: soliciting a DHCP lease client # [ 13.217821] dhcpcd[597]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 13.138916] systemd-logind[508]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 13.228487] dhcpcd[597]: eth0: probing address 10.0.2.15/24 worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.09 seconds) lb: waiting for unit niks3.service client # [ 13.251660] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 13.255072] systemd[1]: Stopped Virtual Console Setup. client # [ 13.276302] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 13.284711] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.162622] systemd-logind[503]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.231158] systemd-vconsole-setup[701]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 13.236120] systemd[1]: Finished Virtual Console Setup. client # [ 13.326929] systemd-logind[486]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.241877] systemd-vconsole-setup[699]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.244630] systemd[1]: Finished Virtual Console Setup. client # [ 13.427204] systemd-vconsole-setup[699]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 13.432852] systemd[1]: Finished Virtual Console Setup. worker2 # [ 13.515150] dhcpcd[637]: eth0: soliciting an IPv6 router worker2 # [ 13.519607] dhcpcd[637]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.522439] dhcpcd[637]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.525434] dhcpcd[637]: eth0: adding route to fec0::/64 worker2 # [ 13.527713] dhcpcd[637]: eth0: adding default route via fe80::2 lb # [ 13.655192] dhcpcd[613]: eth0: soliciting a DHCP lease lb # [ 13.660596] dhcpcd[613]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 13.668220] dhcpcd[613]: eth0: probing address 10.0.2.15/24 lb # [ 13.899607] dhcpcd[613]: eth0: soliciting an IPv6 router lb # [ 13.902639] dhcpcd[613]: eth0: Router Advertisement from fe80::2 lb # [ 13.903508] dhcpcd[613]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 13.909484] dhcpcd[613]: eth0: adding route to fec0::/64 lb # [ 13.910263] dhcpcd[613]: eth0: adding default route via fe80::2 client # [ 14.084836] dhcpcd[597]: eth0: soliciting an IPv6 router client # [ 14.088727] dhcpcd[597]: eth0: Router Advertisement from fe80::2 client # [ 14.091531] dhcpcd[597]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 14.095299] dhcpcd[597]: eth0: adding route to fec0::/64 client # [ 14.097905] dhcpcd[597]: eth0: adding default route via fe80::2 worker1 # [ 14.150440] dhcpcd[632]: eth0: soliciting an IPv6 router worker1 # [ 14.154565] dhcpcd[632]: eth0: Router Advertisement from fe80::2 lb # [ 14.096650] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 14.158159] dhcpcd[632]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.097803] systemd[1]: Stopped Virtual Console Setup. lb # [ 14.098540] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 14.161091] dhcpcd[632]: eth0: adding route to fec0::/64 worker1 # [ 14.163491] dhcpcd[632]: eth0: adding default route via fe80::2 lb # [ 14.104846] systemd[1]: Starting Virtual Console Setup... lb # [ 14.135027] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 14.138292] systemd[1]: Stopped Virtual Console Setup. lb # [ 14.164511] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 14.171360] systemd[1]: Starting Virtual Console Setup... lb # [ 14.223226] systemd-logind[484]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 14.540958] systemd-vconsole-setup[771]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 14.545878] systemd[1]: Finished Virtual Console Setup. lb # [ 14.695617] postgresql-pre-start[695]: selecting default time zone ... UTC lb # [ 14.698839] postgresql-pre-start[695]: creating configuration files ... ok lb # [ 14.926419] postgresql-pre-start[695]: running bootstrap script ... ok lb # [ 15.488322] postgresql-pre-start[695]: performing post-bootstrap initialization ... ok lb # [ 15.655751] postgresql-pre-start[695]: syncing data to disk ... ok lb # [ 15.656916] postgresql-pre-start[695]: initdb: warning: enabling "trust" authentication for local connections lb # [ 15.659337] postgresql-pre-start[695]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 15.661508] postgresql-pre-start[695]: Success. You can now start the database server using: lb # [ 15.662591] postgresql-pre-start[695]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 15.752508] postgres[805]: [805] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 15.755598] postgres[805]: [805] LOG: listening on IPv6 address "::1", port 5432 lb # [ 15.756919] postgres[805]: [805] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 15.759669] postgres[805]: [805] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 15.771612] postgres[814]: [814] LOG: database system was shut down at 2026-09-22 11:25:14 GMT lb # [ 15.776922] postgres[805]: [805] LOG: database system is ready to accept connections lb # [ 15.781951] systemd[1]: Started PostgreSQL Server. lb # [ 15.790065] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 15.957149] postgresql-setup-start[825]: CREATE DATABASE lb # [ 15.995628] postgresql-setup-start[830]: CREATE ROLE lb # [ 16.010471] postgresql-setup-start[832]: ALTER DATABASE lb # [ 16.016414] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 16.018353] systemd[1]: Reached target PostgreSQL. worker2 # [ 16.334794] dhcpcd[637]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 16.337880] dhcpcd[637]: eth0: adding route to 10.0.2.0/24 worker2 # [ 16.343888] dhcpcd[637]: eth0: adding default route via 10.0.2.2 worker2 # [ 16.474025] systemd[1]: Started DHCP Client. worker2 # [ 16.476127] systemd[1]: Reached target Multi-User System. worker2 # [ 16.476935] systemd[1]: Startup finished in 952ms (kernel) + 4.982s (initrd) + 10.540s (userspace) = 16.475s. worker1 # [ 16.704999] dhcpcd[632]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 16.708310] dhcpcd[632]: eth0: adding route to 10.0.2.0/24 worker1 # [ 16.710685] dhcpcd[632]: eth0: adding default route via 10.0.2.2 worker1 # [ 16.844713] systemd[1]: Started DHCP Client. worker1 # [ 16.846865] systemd[1]: Reached target Multi-User System. worker1 # [ 16.847834] systemd[1]: Startup finished in 956ms (kernel) + 4.903s (initrd) + 10.987s (userspace) = 16.846s. lb # [ 17.887089] dhcpcd[613]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.891138] dhcpcd[613]: eth0: adding route to 10.0.2.0/24 lb # [ 17.895304] dhcpcd[613]: eth0: adding default route via 10.0.2.2 lb # [ 18.028767] systemd[1]: Started DHCP Client. lb # [ 18.031207] systemd[1]: Reached target Network is Online. lb # [ 18.036128] systemd[1]: Started Envoy reverse proxy. lb # [ 18.048742] systemd[1]: Started mock-oidc.service. client # [ 18.202979] dhcpcd[597]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 18.206027] dhcpcd[597]: eth0: adding route to 10.0.2.0/24 client # [ 18.210167] dhcpcd[597]: eth0: adding default route via 10.0.2.2 lb # [ 18.127394] mock-oidc-server[909]: Mock OIDC Server running lb # [ 18.129915] mock-oidc-server[909]: OIDC Address: 192.168.1.2:8080 lb # [ 18.130748] mock-oidc-server[909]: Issue Address: 0.0.0.0:8081 lb # [ 18.131530] mock-oidc-server[909]: Issuer: http://192.168.1.2:8080/oidc lb # [ 18.134162] mock-oidc-server[909]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 18.136886] mock-oidc-server[909]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 18.139476] mock-oidc-server[909]: Issue tokens: http://0.0.0.0:8081/issue?sub=... client # [ 18.343762] systemd[1]: Started DHCP Client. client # [ 18.346702] systemd[1]: Reached target Multi-User System. client # [ 18.347675] systemd[1]: Startup finished in 948ms (kernel) + 5.079s (initrd) + 12.318s (userspace) = 18.346s. lb # [ 18.392924] envoy[908]: [2026-09-22 11:25:16.995][908][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 18.394851] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 18.396543] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 18.403520] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 18.411168] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 18.415778] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 18.424995] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 18.426673] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 18.430808] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 18.434875] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 18.440674] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 18.443274] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 18.468431] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 18.471391] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 18.481937] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.484958] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 18.492840] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 18.494651] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 18.496547] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 18.498778] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 18.503758] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 18.506625] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 18.510471] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 18.513141] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 18.519897] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 18.522018] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.524699] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 18.526509] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 18.528687] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 18.531473] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 18.533976] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 18.537325] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 18.539892] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 18.542424] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 18.544964] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 18.547127] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 18.549714] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 18.552037] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 18.557384] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 18.560741] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher worker2 # [ 18.562442] systemd[1]: Reached target Network is Online. lb # [ 18.563230] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.565853] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.567922] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default worker1 # [ 18.629735] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 18.573434] systemd[1]: Starting scheduler-token.service... lb # [ 18.571077] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 18.574651] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 18.577135] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 18.588434] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 18.619021] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 18.621515] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 18.623218] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 18.626258] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 18.627893] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 18.629802] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 18.631568] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 18.633713] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 18.635432] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 18.637426] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 18.639396] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 18.646586] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 18.649570] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 18.656878] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 18.662204] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 18.665255] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 18.667811] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated worker2 # [ 18.669274] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 18.669852] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 18.672521] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift worker2 # [ 18.675348] systemd[1]: Finished scheduler-token.service. lb # [ 18.675810] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 18.678226] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 18.679777] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override worker2 # [ 18.680450] systemd[1]: scheduler-token.service: Consumed 63ms CPU time over 100ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 18.682976] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 18.684848] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation worker2 # [ 18.688467] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.687059] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 18.691719] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 18.693825] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 18.695852] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 18.702244] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 18.704556] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 18.706211] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 18.708141] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 18.711046] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 18.714007] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 18.719251] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 18.722218] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 18.723830] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 18.725968] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log worker1 # [ 18.809311] nix-grpc-daemon[766]: ts=2026-09-22T11:25:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.749958] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 18.753806] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed worker1 # [ 18.817678] nix-grpc-daemon[766]: ts=2026-09-22T11:25:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 18.819646] nix-grpc-daemon[766]: ts=2026-09-22T11:25:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.756265] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based worker1 # [ 18.832179] nix-grpc-daemon[766]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.760683] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy worker1 # [ 18.835118] nix-grpc-daemon[766]: E0000 00:00:1790076317.442839 766 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.774057] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 18.776920] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input worker1 # [ 18.840058] nix-grpc-daemon[766]: E0000 00:00:1790076317.451164 766 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.778774] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 18.783733] envoy[908]: [2026-09-22 11:25:16.996][908][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 18.785825] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 18.787570] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction worker1 # [ 18.860238] nix-grpc-daemon[766]: ts=2026-09-22T11:25:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket lb # [ 18.798331] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 worker1 # [ 18.862690] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 18.800795] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats worker1 # [ 18.868062] nix-grpc-daemon[766]: E0000 00:00:1790076317.478848 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.803675] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 18.813118] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 18.817845] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 18.819736] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 18.822743] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 18.824682] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder worker1 # [ 18.887147] nix-grpc-daemon[766]: E0000 00:00:1790076317.498039 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.826979] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd worker2 # [ 18.833459] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.832293] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 18.834921] envoy[908]: [2026-09-22 11:25:16.997][908][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 18.838312] envoy[908]: [2026-09-22 11:25:17.014][908][info][main] [source/server/server.cc:531] HTTP header map info: worker2 # [ 18.842009] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 18.904155] nix-grpc-daemon[766]: E0000 00:00:1790076317.502061 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.844123] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 18.910045] nix-grpc-daemon[766]: E0000 00:00:1790076317.520435 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.839701] envoy[908]: [2026-09-22 11:25:17.014][908][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id worker2 # [ 18.857582] nix-grpc-daemon[769]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.858414] envoy[908]: [2026-09-22 11:25:17.014][908][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker1 # [ 18.919549] nix-grpc-daemon[766]: ts=2026-09-22T11:25:17Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.860669] nix-grpc-daemon[769]: E0000 00:00:1790076317.471921 769 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.865732] nix-grpc-daemon[769]: E0000 00:00:1790076317.480298 769 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.859929] envoy[908]: [2026-09-22 11:25:17.014][908][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 18.869285] envoy[908]: [2026-09-22 11:25:17.014][908][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 18.871045] envoy[908]: [2026-09-22 11:25:17.059][908][info][main] [source/server/server.cc:968] runtime: {} lb # [ 18.872398] envoy[908]: [2026-09-22 11:25:17.062][908][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 18.874014] envoy[908]: [2026-09-22 11:25:17.062][908][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 18.875549] envoy[908]: [2026-09-22 11:25:17.063][908][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 18.879068] envoy[908]: [2026-09-22 11:25:17.065][908][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 18.882901] envoy[908]: [2026-09-22 11:25:17.065][908][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) worker2 # [ 18.883038] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket lb # [ 18.884835] envoy[908]: [2026-09-22 11:25:17.065][908][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) worker2 # [ 18.887871] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 18.886361] envoy[908]: [2026-09-22 11:25:17.160][908][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 18.887851] envoy[908]: [2026-09-22 11:25:17.177][908][info][config] [source/server/configuration_impl.cc:164] loading stats configuration worker2 # [ 18.892054] nix-grpc-daemon[769]: E0000 00:00:1790076317.506700 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.889908] envoy[908]: [2026-09-22 11:25:17.178][908][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 18.893676] envoy[908]: [2026-09-22 11:25:17.178][908][info][main] [source/server/server.cc:1077] starting main dispatch loop lb # [ 18.895103] envoy[908]: [2026-09-22 11:25:17.417][908][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 18.896838] envoy[908]: [2026-09-22 11:25:17.417][908][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 18.898601] envoy[908]: [2026-09-22 11:25:17.417][908][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 18.900348] envoy[908]: [2026-09-22 11:25:17.461][908][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker2 # [ 18.910891] nix-grpc-daemon[769]: E0000 00:00:1790076317.525525 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.916043] nix-grpc-daemon[769]: E0000 00:00:1790076317.528390 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.932034] nix-grpc-daemon[769]: E0000 00:00:1790076317.546111 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.942011] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.943703] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.120186] nix-grpc-daemon[766]: ts=2026-09-22T11:25:17Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.145235] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.150427] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.351406] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.356294] nix-grpc-daemon[769]: ts=2026-09-22T11:25:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.520774] nix-grpc-daemon[766]: ts=2026-09-22T11:25:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.557405] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.562535] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.763600] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.768765] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.971605] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.975742] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.172743] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.177725] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.323918] nix-grpc-daemon[766]: ts=2026-09-22T11:25:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.327985] nix-grpc-daemon[766]: ts=2026-09-22T11:25:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.378929] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.384123] nix-grpc-daemon[769]: ts=2026-09-22T11:25:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.524835] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.529967] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.585025] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.590450] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.730753] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.735931] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.791897] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.797375] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.936654] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.941648] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.000494] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.004617] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.142373] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.147566] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.201524] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 lb # [ 21.205333] rustfs-bucket-start[947]: mb s3://farm worker2 # [ 21.206836] nix-grpc-daemon[769]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 21.212325] systemd[1]: Finished rustfs-bucket.service. lb # [ 21.219081] systemd[1]: Starting niks3 server... worker1 # [ 21.350437] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.354606] nix-grpc-daemon[766]: ts=2026-09-22T11:25:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 21.378219] postgres[965]: [965] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 21.379482] postgres[965]: [965] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 21.403681] niks3-server[960]: 2026/09/22 11:25:20 OK 20241026095416_initial_model.sql (14.45ms) worker2 # [ 21.408075] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 lb # [ 21.407686] niks3-server[960]: 2026/09/22 11:25:20 OK 20251210153512_drop_unused_gin_index.sql (2.47ms) lb # [ 21.412964] niks3-server[960]: 2026/09/22 11:25:20 OK 20251218171726_add_pins.sql (5.22ms) worker2 # [ 21.413255] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 21.417561] niks3-server[960]: 2026/09/22 11:25:20 OK 20260628120000_add_object_size_and_stats.sql (4.56ms) lb # [ 21.422909] niks3-server[960]: 2026/09/22 11:25:20 OK 20260905000000_add_claims.sql (5.32ms) lb # [ 21.426540] niks3-server[960]: 2026/09/22 11:25:20 OK 20260920000000_drop_claims.sql (3.53ms) lb # [ 21.428446] niks3-server[960]: 2026/09/22 11:25:20 goose: successfully migrated database to version: 20260920000000 lb # [ 21.431899] niks3-server[960]: 2026/09/22 11:25:20 OK 1_commit_pending_closure.sql (5.33ms) lb # [ 21.434536] niks3-server[960]: 2026/09/22 11:25:20 OK 2_object_stats_trigger.sql (2.51ms) lb # [ 21.436398] niks3-server[960]: 2026/09/22 11:25:20 goose: up to current file version: 2 lb # [ 21.438220] niks3-server[960]: 2026/09/22 11:25:20 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 21.462004] niks3-server[960]: 2026/09/22 11:25:20 INFO Read proxy enabled — serving cache objects from S3 lb # [ 21.463315] niks3-server[960]: 2026/09/22 11:25:20 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 21.470829] niks3-server[960]: 2026/09/22 11:25:20 INFO systemd watchdog enabled interval=15s lb # [ 21.471964] niks3-server[960]: 2026/09/22 11:25:20 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 21.474822] systemd[1]: Started niks3 server. lb # [ 21.475677] systemd[1]: Reached target Multi-User System. lb # [ 21.477074] systemd[1]: Startup finished in 965ms (kernel) + 4.154s (initrd) + 16.354s (userspace) = 21.474s. worker1 # [ 21.551111] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.556936] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.614751] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.620612] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.757735] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.762856] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.821688] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.827178] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.963834] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.969315] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.030023] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.034109] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.170209] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.175321] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.230800] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.235877] nix-grpc-daemon[769]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 9.20 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 22.378094] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.382223] nix-grpc-daemon[766]: ts=2026-09-22T11:25:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.05 seconds) lb: waiting for TCP port 50051 on localhost worker2 # [ 22.437188] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.442168] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 22.454815] envoy[908]: conn peer=[::1]:45132 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=4 lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds) worker1 # [ 22.581080] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.598809] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.627025] systemd[1]: Starting scheduler-token.service... worker2 # [ 22.644755] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.648140] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.701797] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 22.706831] systemd[1]: Finished scheduler-token.service. worker2 # [ 22.711316] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 22.791248] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.796627] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 22.849363] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.854531] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.997338] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.002328] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.057086] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.061715] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.203205] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.208363] nix-grpc-daemon[766]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.262836] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.268305] nix-grpc-daemon[769]: ts=2026-09-22T11:25:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.410926] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.415113] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.469442] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.474633] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.611548] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.616763] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.675449] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.680621] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.817514] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.822373] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 23.881739] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.886812] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 24.023171] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.028284] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.089545] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.093638] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.229042] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.234185] nix-grpc-daemon[766]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.290409] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.295553] nix-grpc-daemon[769]: ts=2026-09-22T11:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.436985] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.441109] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.496639] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.501637] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.637288] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.642530] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.702718] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.707870] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.843637] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.849454] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 24.908868] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.915586] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 25.050563] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.055992] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.118755] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.122975] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.256942] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.262114] nix-grpc-daemon[766]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.319549] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.324963] nix-grpc-daemon[769]: ts=2026-09-22T11:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.465194] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.469309] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.526038] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.531227] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.665794] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.671042] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.732559] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.737831] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.871962] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.877050] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.938930] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.944374] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 26.077810] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.082982] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 26.147535] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.151780] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.283821] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.289087] nix-grpc-daemon[766]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.348530] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.353777] nix-grpc-daemon[769]: ts=2026-09-22T11:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.491592] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.495988] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.554938] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.560109] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.691990] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.697407] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.761381] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.767091] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.898287] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.903649] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.967948] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.973076] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 27.104411] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.109822] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 27.176160] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.180198] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.310850] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.317046] nix-grpc-daemon[766]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.376892] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.382018] nix-grpc-daemon[769]: ts=2026-09-22T11:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.519565] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.523821] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.583286] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.588608] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.720111] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.725207] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.789707] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.794777] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.925906] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.932899] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.995727] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.001059] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.130996] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.136240] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 28.203396] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.210059] nix-grpc-daemon[769]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.336868] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.341857] nix-grpc-daemon[766]: ts=2026-09-22T11:25:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.404686] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.409716] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.544765] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.548680] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.610790] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.616514] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.745141] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.750190] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.817439] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.822576] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.951094] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.956465] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.023584] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.028869] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.157231] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.162216] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 29.231951] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.236082] nix-grpc-daemon[769]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 29.363101] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.368298] nix-grpc-daemon[766]: ts=2026-09-22T11:25:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.432864] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.437921] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.570171] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.573429] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.639240] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.644502] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.771272] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.776591] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.845486] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.850748] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.977256] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.982344] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.051725] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.056969] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.183174] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.188377] nix-grpc-daemon[766]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 30.259772] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.263978] nix-grpc-daemon[769]: ts=2026-09-22T11:25:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 30.389138] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.394141] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.460547] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.465601] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.596715] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.600739] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.666753] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.672001] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.796994] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.802007] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.873251] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.879186] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.002785] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.007779] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.079958] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.084985] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.208644] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.218222] nix-grpc-daemon[766]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.287811] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.292078] nix-grpc-daemon[769]: ts=2026-09-22T11:25:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 31.417730] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.422903] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 31.534205] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_take_over lb # [ 31.474342] niks3-server[960]: 2026/09/22 11:25:30 INFO lead: acquired remote=192.168.1.3:49416 worker2 # [ 31.489601] nix-grpc-daemon[769]: ts=2026-09-22T11:25:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.500346] nix-grpc-daemon[769]: ts=2026-09-22T11:25:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.625382] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.629430] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.697332] nix-grpc-daemon[769]: ts=2026-09-22T11:25:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.702601] nix-grpc-daemon[769]: ts=2026-09-22T11:25:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.825986] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.831189] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.903543] nix-grpc-daemon[769]: ts=2026-09-22T11:25:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.987547] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 31.992004] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:55828 worker1 # [ 31.997395] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 32.031906] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.037539] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:55828 worker1 # [ 32.042369] nix-grpc-daemon[766]: ts=2026-09-22T11:25:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.35 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 33.469362] nix-grpc-daemon[766]: ts=2026-09-22T11:25:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 33.484836] systemd[1]: Started Nix Daemon. worker1 # [ 33.669102] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 33.689156] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5) worker1 # [ 33.736692] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 2/5) worker1 # [ 33.758324] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 3/5) worker1 # [ 33.934239] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 4/5) worker1 # [ 33.945644] nix-grpc-daemon[766]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 33.952213] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 33.962645] nix-grpc-daemon[766]: ts=2026-09-22T11:25:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 33.971690] nix-grpc-daemon[766]: ts=2026-09-22T11:25:32Z level=debug event=path_info_miss path=ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test error="error: path '/nix/store/ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 33.937510] nix-grpc-daemon[769]: ts=2026-09-22T11:25:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 33.954004] systemd[1]: Started Nix Daemon. worker2 # [ 34.133864] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 34.153424] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5) worker2 # [ 34.213550] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5) worker2 # [ 34.335365] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 288 ms (attempt 3/5) worker2 # [ 34.625500] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 578 ms (attempt 4/5) worker2 # [ 35.207220] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 35.213261] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 35.223921] nix-grpc-daemon[769]: ts=2026-09-22T11:25:33Z level=debug event=path_info_miss path=ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test error="error: path '/nix/store/ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker2 # [ 35.592082] nix-grpc-daemon[769]: ts=2026-09-22T11:25:34Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 35.595872] nix-grpc-daemon[769]: ts=2026-09-22T11:25:34Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:46832 worker1 # [ 35.675530] nix-grpc-daemon[766]: ts=2026-09-22T11:25:34Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:55844 worker1 # [ 35.688149] nix-grpc-daemon[766]: ts=2026-09-22T11:25:34Z level=debug event=path_info_miss path=ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test error="error: path '/nix/store/ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 35.703038] nix-grpc-daemon[766]: ts=2026-09-22T11:25:34Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:55844 worker1 # [ 35.711041] nix-grpc-daemon[766]: ts=2026-09-22T11:25:34Z level=debug event=path_info_miss path=ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test error="error: path '/nix/store/ba719awibzsjxd1bw9mhq3k18i64cwhc-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds) worker2 # [ 37.910103] nix-grpc-daemon[769]: ts=2026-09-22T11:25:36Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 38.074362] nix-grpc-daemon[769]: ts=2026-09-22T11:25:36Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.95 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 38.301495] nix-grpc-daemon[766]: ts=2026-09-22T11:25:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55844 worker2 # [ 38.242273] nix-grpc-daemon[769]: ts=2026-09-22T11:25:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker1 # [ 38.342099] nix-grpc-daemon[766]: ts=2026-09-22T11:25:36Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 38.347924] nix-grpc-daemon[766]: ts=2026-09-22T11:25:36Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 38.325918] nix-grpc-daemon[769]: ts=2026-09-22T11:25:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 38.353889] nix-grpc-daemon[769]: ts=2026-09-22T11:25:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker2 # [ 38.370351] nix-grpc-daemon[769]: ts=2026-09-22T11:25:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 38.392722] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 38.429647] nix-grpc-daemon[769]: ts=2026-09-22T11:25:37Z level=info event=niks3_push_started pid=850 lb # [ 38.559272] envoy[908]: conn peer=[2001:db8:1::1]:56758 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=1855 worker2 # [ 38.648967] nix-grpc-daemon[850]: time=2026-09-22T11:25:37.263Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 38.760073] nix-daemon[836]: accepted connection from pid 863, user nix-grpc-daemon (trusted) worker2 # [ 38.780237] nix-daemon[836]: reaped child process 866, status = succeeded worker2 # [ 38.883977] nix-daemon[836]: accepted connection from pid 868, user nix-grpc-daemon (trusted) worker2 # [ 38.903402] nix-daemon[836]: reaped child process 871, status = succeeded lb # [ 38.564418] envoy[908]: conn peer=[2001:db8:1::1]:56774 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=86 lb # [ 38.564544] envoy[908]: conn peer=[2001:db8:1::1]:56782 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=12 lb # [ 38.564643] envoy[908]: conn peer=[2001:db8:1::1]:56798 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=13 lb # [ 38.900987] niks3-server[960]: 2026/09/22 11:25:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 38.939909] nix-grpc-daemon[850]: time=2026-09-22T11:25:37.553Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 38.942531] nix-grpc-daemon[850]: time=2026-09-22T11:25:37.553Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 38.986802] niks3-server[960]: 2026/09/22 11:25:37 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 39.002707] niks3-server[960]: 2026/09/22 11:25:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign worker2 # [ 39.012600] nix-grpc-daemon[850]: time=2026-09-22T11:25:37.626Z level=INFO msg="Uploading 1 narinfos" lb # [ 39.014338] niks3-server[960]: 2026/09/22 11:25:37 INFO Signed narinfos id=1 count=1 lb # [ 39.028629] niks3-server[960]: 2026/09/22 11:25:37 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 39.036142] niks3-server[960]: 2026/09/22 11:25:37 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 39.047194] niks3-server[960]: 2026/09/22 11:25:37 INFO Completed upload id=1 worker2 # [ 39.048904] nix-grpc-daemon[850]: time=2026-09-22T11:25:37.663Z level=INFO msg="Upload complete. (396ms)" lb # [ 39.050428] niks3-server[960]: 2026/09/22 11:25:37 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo worker2 # [ 39.061178] nix-grpc-daemon[769]: ts=2026-09-22T11:25:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 39.076854] nix-grpc-daemon[769]: ts=2026-09-22T11:25:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 39.090649] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 39.094418] nix-daemon[836]: reaped child process 848, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... worker2 # [ 39.106553] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker2 # [ 40.214027] nix-daemon[836]: reaped child process 875, status = succeeded worker2 # [ 40.286141] nix-daemon[836]: accepted connection from pid 891, user nix-grpc-daemon (trusted) worker2 # [ 40.303041] nix-daemon[836]: reaped child process 894, status = succeeded lb # [ 40.310195] niks3-server[960]: 2026/09/22 11:25:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 40.324467] nix-grpc-daemon[850]: time=2026-09-22T11:25:38.937Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 40.326256] nix-grpc-daemon[850]: time=2026-09-22T11:25:38.937Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 40.357009] niks3-server[960]: 2026/09/22 11:25:38 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 40.368752] niks3-server[960]: 2026/09/22 11:25:38 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 40.381275] niks3-server[960]: 2026/09/22 11:25:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign worker2 # [ 40.388113] nix-grpc-daemon[850]: time=2026-09-22T11:25:39.001Z level=INFO msg="Uploading 1 narinfos" lb # [ 40.391300] niks3-server[960]: 2026/09/22 11:25:38 INFO Signed narinfos id=2 count=1 lb # [ 40.396763] niks3-server[960]: 2026/09/22 11:25:38 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 40.409752] niks3-server[960]: 2026/09/22 11:25:39 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 40.421517] niks3-server[960]: 2026/09/22 11:25:39 INFO Completed upload id=2 worker2 # [ 40.421310] nix-grpc-daemon[850]: time=2026-09-22T11:25:39.036Z level=INFO msg="Upload complete. (299ms)" lb # [ 40.426386] niks3-server[960]: 2026/09/22 11:25:39 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker2 # [ 40.433178] nix-grpc-daemon[769]: ts=2026-09-22T11:25:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker1 # [ 40.503062] nix-grpc-daemon[766]: ts=2026-09-22T11:25:39Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker1 # [ 40.509015] nix-grpc-daemon[766]: ts=2026-09-22T11:25:39Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 40.453573] nix-daemon[836]: reaped child process 873, status = succeeded worker2 # [ 40.486366] nix-grpc-daemon[769]: ts=2026-09-22T11:25:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 40.501308] nix-grpc-daemon[769]: ts=2026-09-22T11:25:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 40.515850] nix-grpc-daemon[850]: time=2026-09-22T11:25:39.130Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 40.525721] nix-grpc-daemon[769]: ts=2026-09-22T11:25:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker2 # [ 40.547286] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 40.710635] nix-daemon[836]: accepted connection from pid 898, user nix-grpc-daemon (trusted) worker2 # [ 40.736175] nix-daemon[836]: reaped child process 901, status = succeeded worker2 # [ 40.832416] nix-daemon[836]: accepted connection from pid 903, user nix-grpc-daemon (trusted) worker2 # [ 40.849109] nix-daemon[836]: reaped child process 906, status = succeeded lb # [ 40.852973] niks3-server[960]: 2026/09/22 11:25:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 40.863429] nix-grpc-daemon[850]: time=2026-09-22T11:25:39.477Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 40.865177] nix-grpc-daemon[850]: time=2026-09-22T11:25:39.477Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 40.884765] niks3-server[960]: 2026/09/22 11:25:39 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst worker2 # [ 40.894817] nix-grpc-daemon[850]: time=2026-09-22T11:25:39.509Z level=INFO msg="Uploading 1 narinfos" lb # [ 40.899789] niks3-server[960]: 2026/09/22 11:25:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 40.909703] niks3-server[960]: 2026/09/22 11:25:39 INFO Signed narinfos id=3 count=1 lb # [ 40.918400] niks3-server[960]: 2026/09/22 11:25:39 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls worker2 # [ 40.925399] nix-grpc-daemon[850]: time=2026-09-22T11:25:39.540Z level=INFO msg="Upload complete. (360ms)" lb # [ 40.926594] niks3-server[960]: 2026/09/22 11:25:39 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 40.931760] niks3-server[960]: 2026/09/22 11:25:39 INFO Completed upload id=3 worker2 # [ 40.932290] nix-grpc-daemon[769]: ts=2026-09-22T11:25:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 40.935372] niks3-server[960]: 2026/09/22 11:25:39 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker2 # [ 40.945033] nix-grpc-daemon[769]: ts=2026-09-22T11:25:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 40.953359] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 40.957381] nix-daemon[836]: reaped child process 896, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... worker2 # [ 40.968176] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker2 # [ 42.069854] nix-daemon[836]: reaped child process 910, status = succeeded worker2 # [ 42.150554] nix-daemon[836]: accepted connection from pid 927, user nix-grpc-daemon (trusted) worker2 # [ 42.166836] nix-daemon[836]: reaped child process 930, status = succeeded lb # [ 42.173063] niks3-server[960]: 2026/09/22 11:25:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 42.186337] nix-grpc-daemon[850]: time=2026-09-22T11:25:40.800Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 42.188083] nix-grpc-daemon[850]: time=2026-09-22T11:25:40.800Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 42.204420] niks3-server[960]: 2026/09/22 11:25:40 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 42.217627] niks3-server[960]: 2026/09/22 11:25:40 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst worker2 # [ 42.241701] nix-grpc-daemon[850]: time=2026-09-22T11:25:40.856Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.233101] niks3-server[960]: 2026/09/22 11:25:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 42.244977] niks3-server[960]: 2026/09/22 11:25:40 INFO Signed narinfos id=4 count=1 lb # [ 42.247471] niks3-server[960]: 2026/09/22 11:25:40 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 42.265064] niks3-server[960]: 2026/09/22 11:25:40 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 42.273968] niks3-server[960]: 2026/09/22 11:25:40 INFO Completed upload id=4 worker2 # [ 42.275514] nix-grpc-daemon[850]: time=2026-09-22T11:25:40.890Z level=INFO msg="Upload complete. (295ms)" lb # [ 42.276728] niks3-server[960]: 2026/09/22 11:25:40 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 42.355440] nix-grpc-daemon[766]: ts=2026-09-22T11:25:40Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker2 # [ 42.289213] nix-grpc-daemon[769]: ts=2026-09-22T11:25:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 42.361256] nix-grpc-daemon[766]: ts=2026-09-22T11:25:40Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 42.307303] nix-daemon[836]: reaped child process 908, status = succeeded worker2 # [ 42.338197] nix-grpc-daemon[769]: ts=2026-09-22T11:25:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 42.353070] nix-grpc-daemon[769]: ts=2026-09-22T11:25:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 42.367341] nix-grpc-daemon[850]: time=2026-09-22T11:25:40.982Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker2 # [ 42.377563] nix-grpc-daemon[769]: ts=2026-09-22T11:25:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 42.395584] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 42.561937] nix-daemon[836]: accepted connection from pid 934, user nix-grpc-daemon (trusted) worker2 # [ 42.578722] nix-daemon[836]: reaped child process 937, status = succeeded worker2 # [ 42.682217] nix-daemon[836]: accepted connection from pid 939, user nix-grpc-daemon (trusted) worker2 # [ 42.697830] nix-daemon[836]: reaped child process 942, status = succeeded lb # [ 42.703370] niks3-server[960]: 2026/09/22 11:25:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 42.717182] nix-grpc-daemon[850]: time=2026-09-22T11:25:41.331Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 42.718698] nix-grpc-daemon[850]: time=2026-09-22T11:25:41.331Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 42.734231] niks3-server[960]: 2026/09/22 11:25:41 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst worker2 # [ 42.759965] nix-grpc-daemon[850]: time=2026-09-22T11:25:41.374Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.751331] niks3-server[960]: 2026/09/22 11:25:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 42.765624] niks3-server[960]: 2026/09/22 11:25:41 INFO Signed narinfos id=5 count=1 lb # [ 42.772632] niks3-server[960]: 2026/09/22 11:25:41 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 42.786093] niks3-server[960]: 2026/09/22 11:25:41 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete worker2 # [ 42.793086] nix-grpc-daemon[850]: time=2026-09-22T11:25:41.407Z level=INFO msg="Upload complete. (380ms)" lb # [ 42.792828] niks3-server[960]: 2026/09/22 11:25:41 INFO Completed upload id=5 lb # [ 42.799360] niks3-server[960]: 2026/09/22 11:25:41 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker2 # [ 42.804333] nix-grpc-daemon[769]: ts=2026-09-22T11:25:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 42.817773] nix-grpc-daemon[769]: ts=2026-09-22T11:25:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 42.824336] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 42.829318] nix-daemon[836]: reaped child process 932, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... worker2 # [ 42.841813] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker2 # [ 43.917186] nix-daemon[836]: reaped child process 946, status = succeeded worker2 # [ 44.025094] nix-daemon[836]: accepted connection from pid 962, user nix-grpc-daemon (trusted) worker2 # [ 44.043861] nix-daemon[836]: reaped child process 965, status = succeeded lb # [ 44.049033] niks3-server[960]: 2026/09/22 11:25:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 44.060503] nix-grpc-daemon[850]: time=2026-09-22T11:25:42.674Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 44.062354] nix-grpc-daemon[850]: time=2026-09-22T11:25:42.674Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 44.097125] niks3-server[960]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst worker2 # [ 44.108867] nix-grpc-daemon[850]: time=2026-09-22T11:25:42.723Z level=INFO msg="Uploading 1 narinfos" lb # [ 44.109721] niks3-server[960]: 2026/09/22 11:25:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 44.119929] niks3-server[960]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 44.130977] niks3-server[960]: 2026/09/22 11:25:42 INFO Signed narinfos id=6 count=1 worker2 # [ 44.140344] nix-grpc-daemon[850]: time=2026-09-22T11:25:42.755Z level=INFO msg="Upload complete. (288ms)" lb # [ 44.142343] niks3-server[960]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls worker2 # [ 44.152643] nix-grpc-daemon[769]: ts=2026-09-22T11:25:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 lb # [ 44.153263] niks3-server[960]: 2026/09/22 11:25:42 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 44.161009] niks3-server[960]: 2026/09/22 11:25:42 INFO Completed upload id=6 lb # [ 44.164517] niks3-server[960]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker2 # [ 44.170464] nix-daemon[836]: reaped child process 944, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.11 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.64 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker2 # [ 45.394959] nix-grpc-daemon[769]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46832 client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked): client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # [ 45.406987] nix-grpc-daemon[769]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 45.418842] nix-grpc-daemon[850]: time=2026-09-22T11:25:44.032Z level=INFO msg="All 2 paths already cached" worker1 # [ 45.498413] nix-grpc-daemon[766]: ts=2026-09-22T11:25:44Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.28 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.12 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.12 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 worker1 # [ 46.291734] nix-grpc-daemon[766]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55844 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 46.241147] nix-grpc-daemon[769]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 46.251288] nix-grpc-daemon[850]: time=2026-09-22T11:25:44.866Z level=INFO msg="All 2 paths already cached" worker1 # [ 46.327858] nix-grpc-daemon[766]: ts=2026-09-22T11:25:44Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 46.333978] nix-grpc-daemon[766]: ts=2026-09-22T11:25:44Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 46.313817] nix-grpc-daemon[769]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 46.329144] nix-grpc-daemon[769]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 46.344657] nix-grpc-daemon[850]: time=2026-09-22T11:25:44.959Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker2 # [ 46.354850] nix-grpc-daemon[769]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 46.370077] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 46.533732] nix-daemon[836]: accepted connection from pid 983, user nix-grpc-daemon (trusted) worker2 # [ 46.557790] nix-daemon[836]: reaped child process 986, status = succeeded worker2 # [ 46.652107] nix-daemon[836]: accepted connection from pid 988, user nix-grpc-daemon (trusted) worker2 # [ 46.670767] nix-daemon[836]: reaped child process 991, status = succeeded lb # [ 46.673589] niks3-server[960]: 2026/09/22 11:25:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 46.684904] nix-grpc-daemon[850]: time=2026-09-22T11:25:45.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 46.686337] nix-grpc-daemon[850]: time=2026-09-22T11:25:45.299Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 46.702709] niks3-server[960]: 2026/09/22 11:25:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker2 # [ 46.712658] nix-grpc-daemon[850]: time=2026-09-22T11:25:45.327Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.714554] niks3-server[960]: 2026/09/22 11:25:45 INFO Signed narinfos id=7 count=1 lb # [ 46.727289] niks3-server[960]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 46.739216] niks3-server[960]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst lb # [ 46.754197] niks3-server[960]: 2026/09/22 11:25:45 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 46.762311] niks3-server[960]: 2026/09/22 11:25:45 INFO Completed upload id=7 worker2 # [ 46.763853] nix-grpc-daemon[850]: time=2026-09-22T11:25:45.378Z level=INFO msg="Upload complete. (375ms)" lb # [ 46.766254] niks3-server[960]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker2 # [ 46.773155] nix-grpc-daemon[769]: ts=2026-09-22T11:25:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 46.790618] nix-grpc-daemon[769]: ts=2026-09-22T11:25:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 46.800335] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 46.804822] nix-daemon[836]: reaped child process 981, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 46.819737] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker2 # [ 47.660379] nix-daemon[836]: reaped child process 995, status = succeeded worker2 # [ 47.763134] nix-daemon[836]: accepted connection from pid 1011, user nix-grpc-daemon (trusted) worker2 # [ 47.782658] nix-daemon[836]: reaped child process 1014, status = succeeded lb # [ 47.786939] niks3-server[960]: 2026/09/22 11:25:46 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 47.800402] nix-grpc-daemon[850]: time=2026-09-22T11:25:46.414Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 47.801918] nix-grpc-daemon[850]: time=2026-09-22T11:25:46.414Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 47.830150] niks3-server[960]: 2026/09/22 11:25:46 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 47.843550] niks3-server[960]: 2026/09/22 11:25:46 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 47.855980] nix-grpc-daemon[850]: time=2026-09-22T11:25:46.470Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.860273] niks3-server[960]: 2026/09/22 11:25:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 47.868825] niks3-server[960]: 2026/09/22 11:25:46 INFO Signed narinfos id=8 count=1 lb # [ 47.871529] niks3-server[960]: 2026/09/22 11:25:46 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 47.883755] niks3-server[960]: 2026/09/22 11:25:46 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 47.888477] niks3-server[960]: 2026/09/22 11:25:46 INFO Completed upload id=8 worker2 # [ 47.884550] nix-grpc-daemon[850]: time=2026-09-22T11:25:46.499Z level=INFO msg="Upload complete. (255ms)" lb # [ 47.891716] niks3-server[960]: 2026/09/22 11:25:46 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker2 # [ 47.897501] nix-grpc-daemon[769]: ts=2026-09-22T11:25:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker2 # [ 47.916135] nix-daemon[836]: reaped child process 993, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.93 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.44 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 48.498448] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 89740d3a28fe4bd8806530d946f05435 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup lb # [ 48.558900] envoy[908]: conn peer=[2001:db8:1::1]:56800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9506 tx=4080 ms=5940 worker2 # [ 48.563119] nix-grpc-daemon[769]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46832 lb # [ 48.565201] envoy[908]: conn peer=[2001:db8:1::1]:48122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=65 client # [ 48.716216] nix[935]: don't know how to build these paths: client # [ 48.717356] nix[935]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 48.571148] envoy[908]: conn peer=[2001:db8:1::1]:48134 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1809 ms=1682 worker1 # [ 48.651067] nix-grpc-daemon[766]: ts=2026-09-22T11:25:47Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 48.656891] nix-grpc-daemon[766]: ts=2026-09-22T11:25:47Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 48.663785] nix-grpc-daemon[766]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 48.678654] nix-grpc-daemon[766]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 client # [ 48.779458] nix[935]: copying 1 paths to 'lb:50051'... client # [ 48.781468] nix[935]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 48.697878] nix-grpc-daemon[766]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 48.716685] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 48.744734] nix-grpc-daemon[766]: ts=2026-09-22T11:25:47Z level=info event=niks3_push_started pid=902 worker1 # [ 48.872115] nix-grpc-daemon[766]: E0000 00:00:1790076347.480778 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.910995] nix-grpc-daemon[766]: E0000 00:00:1790076347.520688 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 48.940239] nix-grpc-daemon[766]: E0000 00:00:1790076347.527329 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.968203] nix-grpc-daemon[766]: E0000 00:00:1790076347.563740 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 48.992997] nix-grpc-daemon[902]: time=2026-09-22T11:25:47.602Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 49.119275] nix-daemon[847]: accepted connection from pid 915, user nix-grpc-daemon (trusted) worker1 # [ 49.143913] nix-daemon[847]: reaped child process 918, status = succeeded worker1 # [ 49.268173] nix-daemon[847]: accepted connection from pid 920, user nix-grpc-daemon (trusted) worker1 # [ 49.287653] nix-daemon[847]: reaped child process 923, status = succeeded lb # [ 49.235402] niks3-server[960]: 2026/09/22 11:25:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 49.309816] nix-grpc-daemon[902]: time=2026-09-22T11:25:47.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 49.311721] nix-grpc-daemon[902]: time=2026-09-22T11:25:47.919Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" worker1 # [ 49.359037] nix-grpc-daemon[902]: time=2026-09-22T11:25:47.969Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.283208] niks3-server[960]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 49.302112] niks3-server[960]: 2026/09/22 11:25:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 49.311676] niks3-server[960]: 2026/09/22 11:25:47 INFO Signed narinfos id=9 count=1 lb # [ 49.320970] niks3-server[960]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 49.333489] niks3-server[960]: 2026/09/22 11:25:47 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 49.348399] niks3-server[960]: 2026/09/22 11:25:47 INFO Completed upload id=9 worker1 # [ 49.411650] nix-grpc-daemon[902]: time=2026-09-22T11:25:48.022Z level=INFO msg="Upload complete. (415ms)" lb # [ 49.352168] niks3-server[960]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 49.424617] nix-grpc-daemon[766]: ts=2026-09-22T11:25:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 49.444666] nix-grpc-daemon[766]: ts=2026-09-22T11:25:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 49.463587] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 49.472614] nix-daemon[847]: reaped child process 900, status = succeeded worker1 # [ 49.486390] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # [ 49.569060] nix[935]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client # [ 49.625246] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: c79d39bd783b4eafb3eb7432ac8b7c06 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.14 seconds) client # [ 49.905857] nix[951]: don't know how to build these paths: client # [ 49.907650] nix[951]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 49.761916] nix-grpc-daemon[769]: ts=2026-09-22T11:25:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46832 worker1 # [ 49.846356] nix-grpc-daemon[766]: ts=2026-09-22T11:25:48Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 client # [ 49.938710] nix[951]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 49.854852] nix-grpc-daemon[766]: ts=2026-09-22T11:25:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.06 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 53.145851] nix-daemon[847]: reaped child process 927, status = succeeded worker1 # [ 53.251556] nix-daemon[847]: accepted connection from pid 959, user nix-grpc-daemon (trusted) worker1 # [ 53.272785] nix-daemon[847]: reaped child process 962, status = succeeded lb # [ 53.217621] niks3-server[960]: 2026/09/22 11:25:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 53.296972] nix-grpc-daemon[902]: time=2026-09-22T11:25:51.903Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 53.298636] nix-grpc-daemon[902]: time=2026-09-22T11:25:51.903Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 53.261020] niks3-server[960]: 2026/09/22 11:25:51 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 53.299308] niks3-server[960]: 2026/09/22 11:25:51 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 53.317581] niks3-server[960]: 2026/09/22 11:25:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign worker1 # [ 53.380157] nix-grpc-daemon[902]: time=2026-09-22T11:25:51.989Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.326434] niks3-server[960]: 2026/09/22 11:25:51 INFO Signed narinfos id=10 count=1 lb # [ 53.331811] niks3-server[960]: 2026/09/22 11:25:51 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 53.342060] niks3-server[960]: 2026/09/22 11:25:51 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 53.350537] niks3-server[960]: 2026/09/22 11:25:51 INFO Completed upload id=10 worker1 # [ 53.411967] nix-grpc-daemon[902]: time=2026-09-22T11:25:52.022Z level=INFO msg="Upload complete. (307ms)" lb # [ 53.353569] niks3-server[960]: 2026/09/22 11:25:51 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 53.429515] nix-grpc-daemon[766]: ts=2026-09-22T11:25:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 worker1 # [ 53.448567] nix-grpc-daemon[766]: ts=2026-09-22T11:25:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 worker1 # [ 53.457991] nix-daemon[847]: reaped child process 925, status = succeeded client # [ 53.567679] systemd[1]: dedup1.service: Deactivated successfully. client # [ 53.576424] systemd[1]: dedup1.service: Consumed 195ms CPU time over 5.076s wall clock time, 39.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. client # [ 53.603802] systemd[1]: dedup2.service: Deactivated successfully. client # [ 53.606915] systemd[1]: dedup2.service: Consumed 249ms CPU time over 3.982s wall clock time, 33.2M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.20 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 54.782931] nix-grpc-daemon[766]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 54.798545] nix-grpc-daemon[766]: ts=2026-09-22T11:25:53Z level=debug event=want client=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker1 # [ 54.801741] nix-grpc-daemon[766]: ts=2026-09-22T11:25:53Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 54.785230] nix-grpc-daemon[769]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 54.793248] nix-grpc-daemon[769]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 54.804311] nix-grpc-daemon[769]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 54.812437] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 54.957709] nix-daemon[836]: accepted connection from pid 1039, user nix-grpc-daemon (trusted) worker2 # [ 54.986384] nix-daemon[836]: reaped child process 1042, status = succeeded worker2 # [ 55.113823] nix-daemon[836]: accepted connection from pid 1044, user nix-grpc-daemon (trusted) worker2 # [ 55.141214] nix-daemon[836]: reaped child process 1047, status = succeeded lb # [ 55.141648] niks3-server[960]: 2026/09/22 11:25:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 55.154083] nix-grpc-daemon[850]: time=2026-09-22T11:25:53.767Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 55.155538] nix-grpc-daemon[850]: time=2026-09-22T11:25:53.768Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 55.165921] niks3-server[960]: 2026/09/22 11:25:53 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker2 # [ 55.171947] nix-grpc-daemon[850]: time=2026-09-22T11:25:53.786Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.171921] niks3-server[960]: 2026/09/22 11:25:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 55.175921] niks3-server[960]: 2026/09/22 11:25:53 INFO Signed narinfos id=11 count=1 lb # [ 55.179571] niks3-server[960]: 2026/09/22 11:25:53 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 55.185614] niks3-server[960]: 2026/09/22 11:25:53 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 55.188724] niks3-server[960]: 2026/09/22 11:25:53 INFO Completed upload id=11 worker2 # [ 55.189952] nix-grpc-daemon[850]: time=2026-09-22T11:25:53.804Z level=INFO msg="Upload complete. (367ms)" lb # [ 55.191965] niks3-server[960]: 2026/09/22 11:25:53 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 55.195502] nix-grpc-daemon[769]: ts=2026-09-22T11:25:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 55.206883] nix-grpc-daemon[769]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 55.213602] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 55.217818] nix-daemon[836]: reaped child process 1037, status = succeeded worker2 # [ 55.230635] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) lb # [ 58.560443] envoy[908]: conn peer=[2001:db8:1::1]:48136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1451 ms=4824 lb # [ 58.563304] envoy[908]: conn peer=[2001:db8:1::1]:48140 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=3670 worker2 # [ 61.247346] nix-grpc-daemon[769]: ts=2026-09-22T11:25:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=6 outputs=0 worker2 # [ 61.284763] nix-daemon[836]: reaped child process 1049, status = succeeded worker2 # [ 61.286750] nix-daemon[836]: reaped child process 1051, status = succeeded worker2 # [ 61.527200] nix-grpc-daemon[769]: ts=2026-09-22T11:26:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46832 worker1 # [ 61.602873] nix-grpc-daemon[766]: ts=2026-09-22T11:26:00Z level=debug event=want client=7 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 61.603328] nix-grpc-daemon[766]: ts=2026-09-22T11:26:00Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 61.616593] nix-grpc-daemon[766]: ts=2026-09-22T11:26:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 61.632297] nix-grpc-daemon[766]: ts=2026-09-22T11:26:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 61.653031] nix-grpc-daemon[766]: ts=2026-09-22T11:26:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 61.667978] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 61.828379] nix-daemon[847]: accepted connection from pid 979, user nix-grpc-daemon (trusted) worker1 # [ 61.853585] nix-daemon[847]: reaped child process 982, status = succeeded worker1 # [ 61.962876] nix-daemon[847]: accepted connection from pid 984, user nix-grpc-daemon (trusted) worker1 # [ 61.987857] nix-daemon[847]: reaped child process 987, status = succeeded lb # [ 61.930844] niks3-server[960]: 2026/09/22 11:26:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 62.002231] nix-grpc-daemon[902]: time=2026-09-22T11:26:00.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 62.003827] nix-grpc-daemon[902]: time=2026-09-22T11:26:00.612Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 61.956056] niks3-server[960]: 2026/09/22 11:26:00 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 61.972764] niks3-server[960]: 2026/09/22 11:26:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign worker1 # [ 62.039825] nix-grpc-daemon[902]: time=2026-09-22T11:26:00.650Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.979681] niks3-server[960]: 2026/09/22 11:26:00 INFO Signed narinfos id=12 count=1 lb # [ 61.984694] niks3-server[960]: 2026/09/22 11:26:00 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 61.995282] niks3-server[960]: 2026/09/22 11:26:00 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete worker1 # [ 62.064115] nix-grpc-daemon[902]: time=2026-09-22T11:26:00.675Z level=INFO msg="Upload complete. (381ms)" lb # [ 62.003115] niks3-server[960]: 2026/09/22 11:26:00 INFO Completed upload id=12 worker1 # [ 62.070046] nix-grpc-daemon[766]: ts=2026-09-22T11:26:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 62.009178] niks3-server[960]: 2026/09/22 11:26:00 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 62.076368] nix-grpc-daemon[766]: ts=2026-09-22T11:26:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 62.088366] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 62.090516] nix-daemon[847]: reaped child process 977, status = succeeded worker1 # [ 62.101960] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 68.117363] nix-grpc-daemon[766]: ts=2026-09-22T11:26:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=7 outputs=0 worker1 # [ 68.150162] nix-daemon[847]: reaped child process 989, status = succeeded worker1 # [ 68.152376] nix-daemon[847]: reaped child process 991, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.74 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 68.438385] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 267c907587dc48589b5fbf051ed8e7c4 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share worker1 # [ 68.567908] nix-grpc-daemon[766]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55844 client # [ 68.658027] nix[1025]: don't know how to build these paths: client # [ 68.659149] nix[1025]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 68.587623] nix-grpc-daemon[766]: ts=2026-09-22T11:26:07Z level=debug event=want client=8 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 68.594329] nix-grpc-daemon[766]: ts=2026-09-22T11:26:07Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 68.537950] nix-grpc-daemon[769]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 68.555819] nix-grpc-daemon[769]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 lb # [ 68.557693] envoy[908]: conn peer=[2001:db8:1::1]:55324 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=950 ms=6590 lb # [ 68.563074] envoy[908]: conn peer=[2001:db8:1::1]:33790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=976 ms=6555 client # [ 68.720709] nix[1025]: copying 1 paths to 'lb:50051'... client # [ 68.725518] nix[1025]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 68.583079] nix-grpc-daemon[769]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 68.597355] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 68.771358] nix-daemon[836]: accepted connection from pid 1074, user nix-grpc-daemon (trusted) worker2 # [ 68.796524] nix-daemon[836]: reaped child process 1077, status = succeeded worker2 # [ 68.900332] nix-daemon[836]: accepted connection from pid 1079, user nix-grpc-daemon (trusted) worker2 # [ 68.922854] nix-daemon[836]: reaped child process 1082, status = succeeded lb # [ 68.923324] niks3-server[960]: 2026/09/22 11:26:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 68.933701] nix-grpc-daemon[850]: time=2026-09-22T11:26:07.547Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 68.935225] nix-grpc-daemon[850]: time=2026-09-22T11:26:07.547Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 68.969278] niks3-server[960]: 2026/09/22 11:26:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 68.995855] niks3-server[960]: 2026/09/22 11:26:07 INFO Signed narinfos id=13 count=1 worker2 # [ 68.997559] nix-grpc-daemon[850]: time=2026-09-22T11:26:07.612Z level=INFO msg="Uploading 1 narinfos" lb # [ 69.005832] niks3-server[960]: 2026/09/22 11:26:07 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 69.104683] niks3-server[960]: 2026/09/22 11:26:07 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 69.123062] niks3-server[960]: 2026/09/22 11:26:07 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 69.139634] niks3-server[960]: 2026/09/22 11:26:07 INFO Completed upload id=13 worker2 # [ 69.140805] nix-grpc-daemon[850]: time=2026-09-22T11:26:07.755Z level=INFO msg="Upload complete. (525ms)" lb # [ 69.147744] niks3-server[960]: 2026/09/22 11:26:07 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 69.150674] nix-grpc-daemon[769]: ts=2026-09-22T11:26:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 69.172473] nix-grpc-daemon[769]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 69.181932] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 69.189853] nix-daemon[836]: reaped child process 1072, status = succeeded client # [ 69.348231] nix[1025]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker2 # [ 69.209441] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # [ 69.546711] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 2e5737e2f12c4434a89df882368a42fa client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds) worker1 # [ 69.664772] nix-grpc-daemon[766]: ts=2026-09-22T11:26:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55844 client # [ 69.756225] nix[1041]: don't know how to build these paths: client # [ 69.760066] nix[1041]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 69.689518] nix-grpc-daemon[766]: ts=2026-09-22T11:26:08Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 69.633884] nix-grpc-daemon[769]: ts=2026-09-22T11:26:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 client # [ 69.784982] nix[1041]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 72.933116] systemd[1]: share1.service: Sent signal SIGINT to main process 1025 (nix) on client request. client # [ 72.944640] nix[1025]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 72.951513] nix[1025]: [mutex.cc : 451] RAW: Unlock 0xfdfbd69fc788 @ 0xfdffdb1359a4 0xfdffdc20d40c 0xfdfffb6757bc 0xfdfffb677414 0xfdfffb66e1d4 0xfdfffa6b4f7c 0xfdfffa442d80 client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 72.984165] systemd-coredump[1056]: Process 1025 (nix) of user 0 terminated abnormally with signal 11/SEGV, processing... client # [ 73.011966] systemd[1]: Created slice Slice /system/systemd-coredump. client # [ 73.022123] systemd[1]: Started Process Core Dump (PID 1056/UID 0). client # [ 75.417633] systemd-coredump[1061]: Cannot store coredump of 1025 (nix): No space left on device client # [ 75.546543] systemd-coredump[1061]: Process 1025 (nix) of user 0 terminated abnormally without generating a coredump. client # [ 75.564751] systemd[1]: systemd-coredump@0-1-1056_1057-0.service: Deactivated successfully. client # [ 75.566297] systemd[1]: systemd-coredump@0-1-1056_1057-0.service: Consumed 2.060s CPU time over 2.536s wall clock time, 968.8M memory peak, 926.3M written to disk. client # [ 75.574593] systemd[1]: share1.service: Main process exited, code=dumped, status=11/SEGV client # [ 75.577793] systemd[1]: share1.service: Failed with result 'core-dump'. client # [ 75.579034] systemd[1]: share1.service: Consumed 373ms CPU time over 7.139s wall clock time, 43.5M memory peak, 4K written to disk, 6.3K incoming IP traffic, 8.1K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 3.29 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 76.504681] nix-daemon[836]: reaped child process 1086, status = succeeded worker2 # [ 76.603874] nix-daemon[836]: accepted connection from pid 1120, user nix-grpc-daemon (trusted) worker2 # [ 76.626571] nix-daemon[836]: reaped child process 1123, status = succeeded lb # [ 76.625558] niks3-server[960]: 2026/09/22 11:26:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 76.639912] nix-grpc-daemon[850]: time=2026-09-22T11:26:15.252Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 76.642251] nix-grpc-daemon[850]: time=2026-09-22T11:26:15.252Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 76.663301] niks3-server[960]: 2026/09/22 11:26:15 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 76.685355] niks3-server[960]: 2026/09/22 11:26:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker2 # [ 76.692421] nix-grpc-daemon[850]: time=2026-09-22T11:26:15.306Z level=INFO msg="Uploading 1 narinfos" lb # [ 76.695309] niks3-server[960]: 2026/09/22 11:26:15 INFO Signed narinfos id=14 count=1 lb # [ 76.701745] niks3-server[960]: 2026/09/22 11:26:15 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 76.710226] niks3-server[960]: 2026/09/22 11:26:15 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 76.717530] niks3-server[960]: 2026/09/22 11:26:15 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 76.724485] niks3-server[960]: 2026/09/22 11:26:15 INFO Completed upload id=14 worker2 # [ 76.725634] nix-grpc-daemon[850]: time=2026-09-22T11:26:15.340Z level=INFO msg="Upload complete. (296ms)" lb # [ 76.728248] niks3-server[960]: 2026/09/22 11:26:15 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 76.741430] nix-grpc-daemon[769]: ts=2026-09-22T11:26:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=1 worker2 # [ 76.758630] nix-grpc-daemon[769]: ts=2026-09-22T11:26:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=1 worker2 # [ 76.765329] nix-daemon[836]: reaped child process 1084, status = succeeded client # [ 76.910640] systemd[1]: share2.service: Deactivated successfully. client # [ 76.913612] systemd[1]: share2.service: Consumed 181ms CPU time over 7.365s wall clock time, 33.3M memory peak, 4K written to disk, 5.8K incoming IP traffic, 5.5K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.13 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds) (finished: subtest: the first client leaving does not fail the second, in 9.26 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 77.775987] systemd[1]: Started Nix Daemon. client # [ 77.899558] nix-daemon[1107]: accepted connection from pid 1104, user root (trusted) worker2 # [ 77.914063] nix-grpc-daemon[769]: ts=2026-09-22T11:26:16Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:46832 worker1 # [ 77.980441] nix-grpc-daemon[766]: ts=2026-09-22T11:26:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 77.994927] nix-grpc-daemon[766]: ts=2026-09-22T11:26:16Z level=debug event=want client=10 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 78.003068] nix-grpc-daemon[766]: ts=2026-09-22T11:26:16Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 78.013050] nix-grpc-daemon[766]: ts=2026-09-22T11:26:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 78.025886] nix-grpc-daemon[766]: ts=2026-09-22T11:26:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 78.041863] nix-grpc-daemon[766]: ts=2026-09-22T11:26:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 78.058925] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 78.211764] nix-daemon[847]: accepted connection from pid 1036, user nix-grpc-daemon (trusted) worker1 # [ 78.234534] nix-daemon[847]: reaped child process 1039, status = succeeded worker1 # [ 78.337809] nix-daemon[847]: accepted connection from pid 1041, user nix-grpc-daemon (trusted) worker1 # [ 78.358143] nix-daemon[847]: reaped child process 1044, status = succeeded lb # [ 78.299862] niks3-server[960]: 2026/09/22 11:26:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 78.373032] nix-grpc-daemon[902]: time=2026-09-22T11:26:16.983Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 78.374578] nix-grpc-daemon[902]: time=2026-09-22T11:26:16.983Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 78.328619] niks3-server[960]: 2026/09/22 11:26:16 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 78.342198] niks3-server[960]: 2026/09/22 11:26:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker1 # [ 78.411320] nix-grpc-daemon[902]: time=2026-09-22T11:26:17.022Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.354619] niks3-server[960]: 2026/09/22 11:26:16 INFO Signed narinfos id=15 count=1 lb # [ 78.360739] niks3-server[960]: 2026/09/22 11:26:16 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 78.367659] niks3-server[960]: 2026/09/22 11:26:16 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 78.377626] niks3-server[960]: 2026/09/22 11:26:16 INFO Completed upload id=15 worker1 # [ 78.439951] nix-grpc-daemon[902]: time=2026-09-22T11:26:17.050Z level=INFO msg="Upload complete. (365ms)" lb # [ 78.382245] niks3-server[960]: 2026/09/22 11:26:16 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 78.453232] nix-grpc-daemon[766]: ts=2026-09-22T11:26:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 78.460351] nix-grpc-daemon[766]: ts=2026-09-22T11:26:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 78.470419] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 78.475488] nix-daemon[847]: reaped child process 1034, status = succeeded worker1 # [ 78.490221] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) lb # [ 78.557981] envoy[908]: conn peer=[2001:db8:1::1]:33792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=814 ms=6918 lb # [ 78.563255] envoy[908]: conn peer=[2001:db8:1::1]:33808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1086 ms=7152 worker1 # [ 79.568732] nix-daemon[847]: reaped child process 1048, status = succeeded worker1 # [ 79.673325] nix-daemon[847]: accepted connection from pid 1064, user nix-grpc-daemon (trusted) worker1 # [ 79.696244] nix-daemon[847]: reaped child process 1067, status = succeeded lb # [ 79.638618] niks3-server[960]: 2026/09/22 11:26:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 79.711749] nix-grpc-daemon[902]: time=2026-09-22T11:26:18.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 79.713850] nix-grpc-daemon[902]: time=2026-09-22T11:26:18.321Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 79.678180] niks3-server[960]: 2026/09/22 11:26:18 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 79.688739] niks3-server[960]: 2026/09/22 11:26:18 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst worker1 # [ 79.758807] nix-grpc-daemon[902]: time=2026-09-22T11:26:18.369Z level=INFO msg="Uploading 1 narinfos" lb # [ 79.698405] niks3-server[960]: 2026/09/22 11:26:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 79.708686] niks3-server[960]: 2026/09/22 11:26:18 INFO Signed narinfos id=16 count=1 lb # [ 79.710923] niks3-server[960]: 2026/09/22 11:26:18 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 79.723473] niks3-server[960]: 2026/09/22 11:26:18 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 79.730424] niks3-server[960]: 2026/09/22 11:26:18 INFO Completed upload id=16 worker1 # [ 79.792850] nix-grpc-daemon[902]: time=2026-09-22T11:26:18.403Z level=INFO msg="Upload complete. (292ms)" lb # [ 79.733968] niks3-server[960]: 2026/09/22 11:26:18 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 79.805195] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=9 outputs=1 worker1 # [ 79.826957] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 79.832349] nix-daemon[847]: reaped child process 1046, status = succeeded worker2 # [ 79.781173] nix-grpc-daemon[769]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 79.797056] nix-grpc-daemon[769]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 79.815215] nix-grpc-daemon[769]: ts=2026-09-22T11:26:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 80.149602] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 80.157525] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 80.164696] nix-grpc-daemon[902]: time=2026-09-22T11:26:18.774Z level=INFO msg="All 1 paths already cached" worker1 # [ 80.177579] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=want client=11 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 80.187227] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 80.196162] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 80.203773] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 80.215380] nix-grpc-daemon[902]: time=2026-09-22T11:26:18.826Z level=INFO msg="All 1 paths already cached" worker1 # [ 80.224847] nix-grpc-daemon[766]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 80.239088] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 80.395387] nix-daemon[847]: accepted connection from pid 1073, user nix-grpc-daemon (trusted) worker1 # [ 80.417338] nix-daemon[847]: reaped child process 1076, status = succeeded worker1 # [ 80.521778] nix-daemon[847]: accepted connection from pid 1078, user nix-grpc-daemon (trusted) worker1 # [ 80.541404] nix-daemon[847]: reaped child process 1081, status = succeeded lb # [ 80.483188] niks3-server[960]: 2026/09/22 11:26:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 80.558251] nix-grpc-daemon[902]: time=2026-09-22T11:26:19.169Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 80.559781] nix-grpc-daemon[902]: time=2026-09-22T11:26:19.169Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 80.517400] niks3-server[960]: 2026/09/22 11:26:19 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker1 # [ 80.592823] nix-grpc-daemon[902]: time=2026-09-22T11:26:19.203Z level=INFO msg="Uploading 1 narinfos" lb # [ 80.531200] niks3-server[960]: 2026/09/22 11:26:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign lb # [ 80.543371] niks3-server[960]: 2026/09/22 11:26:19 INFO Signed narinfos id=17 count=1 lb # [ 80.550642] niks3-server[960]: 2026/09/22 11:26:19 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 80.564843] niks3-server[960]: 2026/09/22 11:26:19 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker1 # [ 80.626793] nix-grpc-daemon[902]: time=2026-09-22T11:26:19.237Z level=INFO msg="Upload complete. (372ms)" lb # [ 80.575009] niks3-server[960]: 2026/09/22 11:26:19 INFO Completed upload id=17 worker1 # [ 80.636483] nix-grpc-daemon[766]: ts=2026-09-22T11:26:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 80.578097] niks3-server[960]: 2026/09/22 11:26:19 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 80.650275] nix-grpc-daemon[766]: ts=2026-09-22T11:26:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 80.660000] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 80.663039] nix-daemon[847]: reaped child process 1071, status = succeeded worker1 # [ 80.674329] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 81.944874] nix-daemon[847]: reaped child process 1085, status = succeeded worker1 # [ 82.043439] nix-daemon[847]: accepted connection from pid 1101, user nix-grpc-daemon (trusted) worker1 # [ 82.063931] nix-daemon[847]: reaped child process 1104, status = succeeded lb # [ 82.006465] niks3-server[960]: 2026/09/22 11:26:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 82.080492] nix-grpc-daemon[902]: time=2026-09-22T11:26:20.690Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 82.082064] nix-grpc-daemon[902]: time=2026-09-22T11:26:20.690Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 82.038526] niks3-server[960]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 82.055839] niks3-server[960]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 82.071134] niks3-server[960]: 2026/09/22 11:26:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 82.138819] nix-grpc-daemon[902]: time=2026-09-22T11:26:20.749Z level=INFO msg="Uploading 1 narinfos" lb # [ 82.077825] niks3-server[960]: 2026/09/22 11:26:20 INFO Signed narinfos id=18 count=1 lb # [ 82.085152] niks3-server[960]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 82.096737] niks3-server[960]: 2026/09/22 11:26:20 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete worker1 # [ 82.165123] nix-grpc-daemon[902]: time=2026-09-22T11:26:20.776Z level=INFO msg="Upload complete. (277ms)" lb # [ 82.104924] niks3-server[960]: 2026/09/22 11:26:20 INFO Completed upload id=18 lb # [ 82.109681] niks3-server[960]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 82.180457] nix-grpc-daemon[766]: ts=2026-09-22T11:26:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=10 outputs=1 worker1 # [ 82.200189] nix-grpc-daemon[766]: ts=2026-09-22T11:26:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 82.205462] nix-daemon[847]: reaped child process 1083, status = succeeded worker1 # [ 82.215803] nix-grpc-daemon[766]: ts=2026-09-22T11:26:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 82.221939] nix-grpc-daemon[766]: ts=2026-09-22T11:26:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 82.508269] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:55844 worker2 # [ 82.454715] nix-grpc-daemon[769]: ts=2026-09-22T11:26:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 82.473778] nix-grpc-daemon[850]: time=2026-09-22T11:26:21.087Z level=INFO msg="All 1 paths already cached" worker1 # [ 82.555741] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=debug event=want client=12 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 82.563383] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 82.572811] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 82.585673] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 82.601048] nix-grpc-daemon[902]: time=2026-09-22T11:26:21.212Z level=INFO msg="All 2 paths already cached" worker1 # [ 82.611422] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 82.624656] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 82.781921] nix-daemon[847]: accepted connection from pid 1110, user nix-grpc-daemon (trusted) worker1 # [ 82.805395] nix-daemon[847]: reaped child process 1113, status = succeeded worker1 # [ 82.917130] nix-daemon[847]: accepted connection from pid 1115, user nix-grpc-daemon (trusted) worker1 # [ 82.943059] nix-daemon[847]: reaped child process 1118, status = succeeded lb # [ 82.877258] niks3-server[960]: 2026/09/22 11:26:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 82.950330] nix-grpc-daemon[902]: time=2026-09-22T11:26:21.560Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 82.951872] nix-grpc-daemon[902]: time=2026-09-22T11:26:21.561Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 82.909610] niks3-server[960]: 2026/09/22 11:26:21 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 82.933063] niks3-server[960]: 2026/09/22 11:26:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker1 # [ 83.001737] nix-grpc-daemon[902]: time=2026-09-22T11:26:21.612Z level=INFO msg="Uploading 1 narinfos" lb # [ 82.942645] niks3-server[960]: 2026/09/22 11:26:21 INFO Signed narinfos id=19 count=1 lb # [ 82.952446] niks3-server[960]: 2026/09/22 11:26:21 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 82.967188] niks3-server[960]: 2026/09/22 11:26:21 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete worker1 # [ 83.037923] nix-grpc-daemon[902]: time=2026-09-22T11:26:21.647Z level=INFO msg="Upload complete. (396ms)" lb # [ 82.977774] niks3-server[960]: 2026/09/22 11:26:21 INFO Completed upload id=19 lb # [ 82.981970] niks3-server[960]: 2026/09/22 11:26:21 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 83.049074] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 83.064555] nix-grpc-daemon[766]: ts=2026-09-22T11:26:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 83.075004] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 83.078870] nix-daemon[847]: reaped child process 1108, status = succeeded worker1 # [ 83.091979] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 84.572779] nix-daemon[847]: reaped child process 1122, status = succeeded worker1 # [ 84.680068] nix-daemon[847]: accepted connection from pid 1138, user nix-grpc-daemon (trusted) worker1 # [ 84.703936] nix-daemon[847]: reaped child process 1141, status = succeeded lb # [ 84.641952] niks3-server[960]: 2026/09/22 11:26:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 84.715504] nix-grpc-daemon[902]: time=2026-09-22T11:26:23.324Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 84.717152] nix-grpc-daemon[902]: time=2026-09-22T11:26:23.324Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 84.686429] niks3-server[960]: 2026/09/22 11:26:23 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker1 # [ 84.759473] nix-grpc-daemon[902]: time=2026-09-22T11:26:23.369Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.698997] niks3-server[960]: 2026/09/22 11:26:23 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 84.711293] niks3-server[960]: 2026/09/22 11:26:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 84.723363] niks3-server[960]: 2026/09/22 11:26:23 INFO Signed narinfos id=20 count=1 lb # [ 84.729676] niks3-server[960]: 2026/09/22 11:26:23 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls worker1 # [ 84.791854] nix-grpc-daemon[902]: time=2026-09-22T11:26:23.402Z level=INFO msg="Upload complete. (282ms)" lb # [ 84.735363] niks3-server[960]: 2026/09/22 11:26:23 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 84.741174] niks3-server[960]: 2026/09/22 11:26:23 INFO Completed upload id=20 lb # [ 84.744319] niks3-server[960]: 2026/09/22 11:26:23 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 84.805242] nix-grpc-daemon[766]: ts=2026-09-22T11:26:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=11 outputs=1 worker1 # [ 84.825425] nix-grpc-daemon[766]: ts=2026-09-22T11:26:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 84.830773] nix-daemon[847]: reaped child process 1120, status = succeeded worker2 # [ 84.790555] nix-grpc-daemon[769]: ts=2026-09-22T11:26:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 84.806150] nix-grpc-daemon[769]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 84.822650] nix-grpc-daemon[769]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker2 # [ 84.838249] nix-grpc-daemon[769]: ts=2026-09-22T11:26:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_out=232 client # [ 85.121639] nix-daemon[1107]: reaped child process 1109, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.51 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.63 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.31 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 86.285008] nix-daemon[1107]: accepted connection from pid 1190, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker2 # [ 86.416764] nix-grpc-daemon[769]: ts=2026-09-22T11:26:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:46832 client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 86.424165] nix-grpc-daemon[769]: ts=2026-09-22T11:26:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker2 # [ 86.442620] nix-grpc-daemon[769]: ts=2026-09-22T11:26:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 86.459490] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 86.635198] nix-daemon[836]: accepted connection from pid 1133, user nix-grpc-daemon (trusted) worker2 # [ 86.658424] nix-daemon[836]: reaped child process 1136, status = succeeded lb # [ 86.660594] niks3-server[960]: 2026/09/22 11:26:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 86.672364] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.286Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 86.674070] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.286Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 86.692247] niks3-server[960]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 86.709425] niks3-server[960]: 2026/09/22 11:26:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker2 # [ 86.720748] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.335Z level=INFO msg="Uploading 1 narinfos" lb # [ 86.722994] niks3-server[960]: 2026/09/22 11:26:25 INFO Signed narinfos id=21 count=1 lb # [ 86.731632] niks3-server[960]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 86.744239] niks3-server[960]: 2026/09/22 11:26:25 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete worker2 # [ 86.757517] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.372Z level=INFO msg="Upload complete. (278ms)" lb # [ 86.755964] niks3-server[960]: 2026/09/22 11:26:25 INFO Completed upload id=21 worker2 # [ 86.763134] nix-grpc-daemon[769]: ts=2026-09-22T11:26:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=120 lb # [ 86.762896] niks3-server[960]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker2 # [ 86.773480] nix-daemon[836]: reaped child process 1131, status = succeeded worker1 # [ 86.859796] nix-grpc-daemon[766]: ts=2026-09-22T11:26:25Z level=debug event=want client=13 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 86.867157] nix-grpc-daemon[766]: ts=2026-09-22T11:26:25Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 86.844921] nix-grpc-daemon[769]: ts=2026-09-22T11:26:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 86.866594] nix-grpc-daemon[769]: ts=2026-09-22T11:26:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 client # copying 2 paths to 'lb:50051'... worker2 # [ 86.892109] nix-grpc-daemon[769]: ts=2026-09-22T11:26:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 86.916207] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 87.096420] nix-daemon[836]: accepted connection from pid 1140, user nix-grpc-daemon (trusted) worker2 # [ 87.119319] nix-daemon[836]: reaped child process 1143, status = succeeded worker2 # [ 87.230657] nix-daemon[836]: accepted connection from pid 1145, user nix-grpc-daemon (trusted) worker2 # [ 87.252436] nix-daemon[836]: reaped child process 1148, status = succeeded lb # [ 87.255323] niks3-server[960]: 2026/09/22 11:26:25 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 87.266637] niks3-server[960]: 2026/09/22 11:26:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 87.280114] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.894Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 87.281804] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.894Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" worker2 # [ 87.283652] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.894Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" lb # [ 87.307687] niks3-server[960]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 87.327183] niks3-server[960]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 87.341377] niks3-server[960]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 87.355928] niks3-server[960]: 2026/09/22 11:26:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 87.359055] niks3-server[960]: 2026/09/22 11:26:25 INFO Signed narinfos id=22 count=1 worker2 # [ 87.366311] nix-grpc-daemon[850]: time=2026-09-22T11:26:25.979Z level=INFO msg="Uploading 3 narinfos" lb # [ 87.369380] niks3-server[960]: 2026/09/22 11:26:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 87.377393] niks3-server[960]: 2026/09/22 11:26:25 INFO Signed narinfos id=23 count=2 lb # [ 87.382729] niks3-server[960]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 87.399876] niks3-server[960]: 2026/09/22 11:26:26 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 87.404800] niks3-server[960]: 2026/09/22 11:26:26 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 87.412541] niks3-server[960]: 2026/09/22 11:26:26 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 87.414630] niks3-server[960]: 2026/09/22 11:26:26 INFO Completed upload id=22 lb # [ 87.415827] niks3-server[960]: 2026/09/22 11:26:26 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 87.419490] niks3-server[960]: 2026/09/22 11:26:26 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 87.422603] niks3-server[960]: 2026/09/22 11:26:26 INFO Completed upload id=23 worker2 # [ 87.424488] nix-grpc-daemon[850]: time=2026-09-22T11:26:26.039Z level=INFO msg="Upload complete. (485ms)" worker2 # [ 87.440118] nix-grpc-daemon[769]: ts=2026-09-22T11:26:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 87.461916] nix-grpc-daemon[769]: ts=2026-09-22T11:26:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 87.470701] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 87.478403] nix-daemon[836]: reaped child process 1138, status = succeeded worker2 # [ 87.488922] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 88.562836] envoy[908]: conn peer=[2001:db8:1::1]:49552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3417 tx=1713 ms=1963 worker2 # [ 88.720413] nix-daemon[836]: reaped child process 1152, status = succeeded worker2 # [ 88.855065] nix-daemon[836]: accepted connection from pid 1168, user nix-grpc-daemon (trusted) worker2 # [ 88.878350] nix-daemon[836]: reaped child process 1171, status = succeeded lb # [ 88.568232] envoy[908]: conn peer=[2001:db8:1::1]:49566 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=100 lb # [ 88.568501] envoy[908]: conn peer=[2001:db8:1::1]:49574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3899 tx=1956 ms=2127 lb # [ 88.568611] envoy[908]: conn peer=[2001:db8:1::1]:50766 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=64 lb # [ 88.568706] envoy[908]: conn peer=[2001:db8:1::1]:50780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4338 tx=2188 ms=2454 lb # [ 88.568799] envoy[908]: conn peer=[2001:db8:1::1]:50786 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=116 lb # [ 88.889888] niks3-server[960]: 2026/09/22 11:26:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 88.936631] nix-grpc-daemon[850]: time=2026-09-22T11:26:27.548Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 88.965672] niks3-server[960]: 2026/09/22 11:26:27 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 88.976638] niks3-server[960]: 2026/09/22 11:26:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 88.984581] niks3-server[960]: 2026/09/22 11:26:27 INFO Signed narinfos id=24 count=1 worker2 # [ 88.984680] nix-grpc-daemon[850]: time=2026-09-22T11:26:27.598Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.994320] niks3-server[960]: 2026/09/22 11:26:27 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 89.014462] niks3-server[960]: 2026/09/22 11:26:27 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 89.028700] niks3-server[960]: 2026/09/22 11:26:27 INFO Completed upload id=24 lb # [ 89.032212] niks3-server[960]: 2026/09/22 11:26:27 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker2 # [ 89.030485] nix-grpc-daemon[850]: time=2026-09-22T11:26:27.645Z level=INFO msg="Upload complete. (325ms)" worker2 # [ 89.045906] nix-grpc-daemon[769]: ts=2026-09-22T11:26:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=12 outputs=1 worker2 # [ 89.071288] nix-daemon[836]: reaped child process 1150, status = succeeded client # copying 1 paths... worker1 # [ 89.167760] nix-grpc-daemon[766]: ts=2026-09-22T11:26:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 89.194366] nix-grpc-daemon[766]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker1 # [ 89.277902] nix-grpc-daemon[766]: ts=2026-09-22T11:26:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 89.286023] nix-grpc-daemon[766]: ts=2026-09-22T11:26:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_out=120 client # [ 89.486406] nix-daemon[1107]: reaped child process 1193, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.49 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 89.732549] nix-daemon[1107]: accepted connection from pid 1212, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 89.878726] nix-grpc-daemon[766]: ts=2026-09-22T11:26:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:55844 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 89.826311] nix-grpc-daemon[769]: ts=2026-09-22T11:26:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 89.842526] nix-grpc-daemon[850]: time=2026-09-22T11:26:28.457Z level=INFO msg="All 1 paths already cached" worker1 # [ 89.931781] nix-grpc-daemon[766]: ts=2026-09-22T11:26:28Z level=debug event=want client=14 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 89.941396] nix-grpc-daemon[766]: ts=2026-09-22T11:26:28Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 89.954631] nix-grpc-daemon[766]: ts=2026-09-22T11:26:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 89.971635] nix-grpc-daemon[766]: ts=2026-09-22T11:26:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55844 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 90.016975] nix-grpc-daemon[766]: ts=2026-09-22T11:26:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 90.045998] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 90.084411] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5) worker1 # [ 90.157589] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 2/5) worker1 # [ 90.334796] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 270 ms (attempt 3/5) worker1 # [ 90.607541] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 617 ms (attempt 4/5) worker1 # [ 91.228763] nix-grpc-daemon[766]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 91.236746] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 91.245247] nix-grpc-daemon[766]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 91.558585] nix-daemon[847]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker1 # [ 91.598489] nix-daemon[847]: reaped child process 1163, status = succeeded worker1 # [ 91.695530] nix-daemon[847]: accepted connection from pid 1165, user nix-grpc-daemon (trusted) worker1 # [ 91.716846] nix-daemon[847]: reaped child process 1168, status = succeeded lb # [ 91.655108] niks3-server[960]: 2026/09/22 11:26:30 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 91.731761] nix-grpc-daemon[902]: time=2026-09-22T11:26:30.341Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 91.733481] nix-grpc-daemon[902]: time=2026-09-22T11:26:30.341Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 91.692757] niks3-server[960]: 2026/09/22 11:26:30 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 91.708530] niks3-server[960]: 2026/09/22 11:26:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker1 # [ 91.777333] nix-grpc-daemon[902]: time=2026-09-22T11:26:30.387Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.714853] niks3-server[960]: 2026/09/22 11:26:30 INFO Signed narinfos id=25 count=1 lb # [ 91.723718] niks3-server[960]: 2026/09/22 11:26:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 91.734327] niks3-server[960]: 2026/09/22 11:26:30 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete worker1 # [ 91.804510] nix-grpc-daemon[902]: time=2026-09-22T11:26:30.415Z level=INFO msg="Upload complete. (518ms)" lb # [ 91.745296] niks3-server[960]: 2026/09/22 11:26:30 INFO Completed upload id=25 lb # [ 91.749867] niks3-server[960]: 2026/09/22 11:26:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 91.813077] nix-grpc-daemon[766]: ts=2026-09-22T11:26:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=1 paths=1 nar_bytes_in=568 worker1 # [ 91.826803] nix-grpc-daemon[766]: ts=2026-09-22T11:26:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 91.838806] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 91.848375] nix-daemon[847]: reaped child process 1153, status = succeeded client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker1 # [ 91.857820] nix-daemon[847]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 92.965110] nix-daemon[847]: reaped child process 1172, status = succeeded worker1 # [ 93.035813] nix-daemon[847]: accepted connection from pid 1188, user nix-grpc-daemon (trusted) worker1 # [ 93.058100] nix-daemon[847]: reaped child process 1191, status = succeeded lb # [ 92.994503] niks3-server[960]: 2026/09/22 11:26:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.072147] nix-grpc-daemon[902]: time=2026-09-22T11:26:31.679Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 93.025389] niks3-server[960]: 2026/09/22 11:26:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign worker1 # [ 93.097821] nix-grpc-daemon[902]: time=2026-09-22T11:26:31.708Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.036976] niks3-server[960]: 2026/09/22 11:26:31 INFO Signed narinfos id=26 count=1 lb # [ 93.048030] niks3-server[960]: 2026/09/22 11:26:31 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 93.057097] niks3-server[960]: 2026/09/22 11:26:31 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv lb # [ 93.067813] niks3-server[960]: 2026/09/22 11:26:31 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete worker1 # [ 93.135578] nix-grpc-daemon[902]: time=2026-09-22T11:26:31.746Z level=INFO msg="Upload complete. (268ms)" lb # [ 93.074218] niks3-server[960]: 2026/09/22 11:26:31 INFO Completed upload id=26 lb # [ 93.080796] niks3-server[960]: 2026/09/22 11:26:31 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 93.149138] nix-grpc-daemon[766]: ts=2026-09-22T11:26:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=13 outputs=1 worker1 # [ 93.165208] nix-daemon[847]: reaped child process 1170, status = succeeded client # copying 1 paths... worker2 # [ 93.135145] nix-grpc-daemon[769]: ts=2026-09-22T11:26:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 93.151371] nix-grpc-daemon[769]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 93.263194] nix-grpc-daemon[766]: ts=2026-09-22T11:26:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 93.268597] nix-grpc-daemon[766]: ts=2026-09-22T11:26:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_out=120 client # [ 93.447391] nix-daemon[1107]: reaped child process 1215, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.96 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 93.620322] nix-daemon[1107]: accepted connection from pid 1234, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 93.693356] nix-grpc-daemon[766]: ts=2026-09-22T11:26:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:55844 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 93.636594] nix-grpc-daemon[769]: ts=2026-09-22T11:26:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 93.649342] nix-grpc-daemon[850]: time=2026-09-22T11:26:32.263Z level=INFO msg="All 1 paths already cached" worker1 # [ 93.729047] nix-grpc-daemon[766]: ts=2026-09-22T11:26:32Z level=debug event=want client=15 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker1 # [ 93.735934] nix-grpc-daemon[766]: ts=2026-09-22T11:26:32Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 93.678221] nix-grpc-daemon[769]: ts=2026-09-22T11:26:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 93.694372] nix-grpc-daemon[769]: ts=2026-09-22T11:26:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 93.709830] nix-grpc-daemon[850]: time=2026-09-22T11:26:32.324Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 93.721184] nix-grpc-daemon[769]: ts=2026-09-22T11:26:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 93.740410] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 93.903286] nix-daemon[836]: accepted connection from pid 1176, user nix-grpc-daemon (trusted) worker2 # [ 93.928626] nix-daemon[836]: reaped child process 1179, status = succeeded worker2 # [ 94.026353] nix-daemon[836]: accepted connection from pid 1181, user nix-grpc-daemon (trusted) worker2 # [ 94.042982] nix-daemon[836]: reaped child process 1184, status = succeeded lb # [ 94.050611] niks3-server[960]: 2026/09/22 11:26:32 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 94.061637] nix-grpc-daemon[850]: time=2026-09-22T11:26:32.675Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 94.063128] nix-grpc-daemon[850]: time=2026-09-22T11:26:32.676Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 94.086446] niks3-server[960]: 2026/09/22 11:26:32 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 94.100784] nix-grpc-daemon[850]: time=2026-09-22T11:26:32.715Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.099549] niks3-server[960]: 2026/09/22 11:26:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 94.110278] niks3-server[960]: 2026/09/22 11:26:32 INFO Signed narinfos id=27 count=1 lb # [ 94.115814] niks3-server[960]: 2026/09/22 11:26:32 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 94.124932] niks3-server[960]: 2026/09/22 11:26:32 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 94.129377] niks3-server[960]: 2026/09/22 11:26:32 INFO Completed upload id=27 worker2 # [ 94.130251] nix-grpc-daemon[850]: time=2026-09-22T11:26:32.745Z level=INFO msg="Upload complete. (369ms)" lb # [ 94.131853] niks3-server[960]: 2026/09/22 11:26:32 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 94.141385] nix-grpc-daemon[769]: ts=2026-09-22T11:26:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 94.154557] nix-grpc-daemon[769]: ts=2026-09-22T11:26:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 worker2 # [ 94.162519] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 94.164732] nix-daemon[836]: reaped child process 1174, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 94.177112] nix-daemon[836]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 94.852843] nix-daemon[836]: reaped child process 1188, status = succeeded worker2 # [ 94.952663] nix-daemon[836]: accepted connection from pid 1204, user nix-grpc-daemon (trusted) worker2 # [ 94.972572] nix-daemon[836]: reaped child process 1207, status = succeeded lb # [ 94.974407] niks3-server[960]: 2026/09/22 11:26:33 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 94.986890] nix-grpc-daemon[850]: time=2026-09-22T11:26:33.599Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" worker2 # [ 95.005347] nix-grpc-daemon[850]: time=2026-09-22T11:26:33.619Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.003988] niks3-server[960]: 2026/09/22 11:26:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 95.017094] niks3-server[960]: 2026/09/22 11:26:33 INFO Signed narinfos id=28 count=1 lb # [ 95.026064] niks3-server[960]: 2026/09/22 11:26:33 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 95.038484] niks3-server[960]: 2026/09/22 11:26:33 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker2 # [ 95.046939] nix-grpc-daemon[850]: time=2026-09-22T11:26:33.661Z level=INFO msg="Upload complete. (257ms)" lb # [ 95.047535] niks3-server[960]: 2026/09/22 11:26:33 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 95.056618] niks3-server[960]: 2026/09/22 11:26:33 INFO Completed upload id=28 client # copying 1 paths... lb # [ 95.061772] niks3-server[960]: 2026/09/22 11:26:33 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker2 # [ 95.060284] nix-grpc-daemon[769]: ts=2026-09-22T11:26:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46832 duration_s=0 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=14 outputs=1 worker1 # [ 95.126066] nix-grpc-daemon[766]: ts=2026-09-22T11:26:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:55844 worker2 # [ 95.074863] nix-daemon[836]: reaped child process 1186, status = succeeded worker1 # [ 95.139025] nix-grpc-daemon[766]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 95.181779] nix-grpc-daemon[766]: ts=2026-09-22T11:26:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 95.187016] nix-grpc-daemon[766]: ts=2026-09-22T11:26:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55844 duration_s=0 paths=1 nar_bytes_out=120 client # [ 95.390337] nix-daemon[1107]: reaped child process 1237, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.94 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 10.14 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds) worker2 # [ 96.271386] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 96.916583] nix-grpc-daemon[769]: ts=2026-09-22T11:26:35Z level=info event=signal_exit worker2 # [ 98.477517] nix-grpc-daemon[769]: ts=2026-09-22T11:26:37Z level=info event=scheduler_restarting worker1 # [ 98.552664] nix-grpc-daemon[766]: ts=2026-09-22T11:26:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:55828 worker1 # [ 98.552985] nix-grpc-daemon[766]: ts=2026-09-22T11:26:37Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 98.517025] nix-grpc-daemon[769]: ts=2026-09-22T11:26:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" lb # [ 98.559831] envoy[908]: conn peer=[2001:db8:1::1]:50796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4273 tx=1977 ms=2834 worker2 # [ 98.563967] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 98.567842] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 98.581875] systemd[1]: nix-grpc-daemon.service: Consumed 3.881s CPU time over 1min 19.874s wall clock time, 62.1M memory peak, 91.7K incoming IP traffic, 201.3K outgoing IP traffic. worker2 # [ 98.594910] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 98.597469] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 98.598536] systemd[1]: nix-grpc-daemon.socket: Consumed 622us CPU time over 1min 28.234s wall clock time, 256K memory peak, 85.6K incoming IP traffic, 71K outgoing IP traffic. worker2 # [ 98.602092] nix-daemon[836]: reaped child process 838, status = succeeded lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 98.952314] nix-grpc-daemon[766]: ts=2026-09-22T11:26:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55844 worker1 # [ 98.961460] nix-grpc-daemon[766]: ts=2026-09-22T11:26:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:55844 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 99.017813] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 99.153272] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 99.107220] systemd[1]: Starting scheduler-token.service... lb # [ 98.565316] envoy[908]: conn peer=[2001:db8:1::1]:50804 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=52 lb # [ 98.565453] envoy[908]: conn peer=[2001:db8:1::1]:42418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=27 lb # [ 98.565593] envoy[908]: conn peer=[2001:db8:1::1]:50812 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3772 tx=2030 ms=3411 lb # [ 98.565709] envoy[908]: conn peer=[2001:db8:1::1]:42422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3725 tx=1972 ms=1546 lb # [ 98.565810] envoy[908]: conn peer=[2001:db8:1::1]:42428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=61 lb # [ 98.565910] envoy[908]: conn peer=[2001:db8:1::4]:52832 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=50553 tx=2956 ms=79555 worker2 # [ 99.228188] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 99.236487] systemd[1]: Finished scheduler-token.service. worker2 # [ 99.238743] systemd[1]: scheduler-token.service: Consumed 60ms CPU time over 124ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 99.249761] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 99.396729] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:38Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 99.398742] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 99.403058] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 99.409271] nix-grpc-daemon[1231]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 99.410810] nix-grpc-daemon[1231]: E0000 00:00:1790076398.023426 1231 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 99.456310] nix-grpc-daemon[1231]: E0000 00:00:1790076398.030077 1231 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 99.458691] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:38Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 99.467694] nix-grpc-daemon[1231]: E0000 00:00:1790076398.042659 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 99.470929] nix-grpc-daemon[1231]: E0000 00:00:1790076398.061745 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 99.473587] nix-grpc-daemon[1231]: E0000 00:00:1790076398.061989 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 99.479168] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 99.482564] nix-grpc-daemon[1231]: E0000 00:00:1790076398.081321 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 99.485221] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 99.546974] nix-grpc-daemon[766]: ts=2026-09-22T11:26:38Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:55828 worker1 # [ 99.554094] nix-grpc-daemon[766]: ts=2026-09-22T11:26:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 99.888959] nix-grpc-daemon[766]: ts=2026-09-22T11:26:38Z level=info event=signal_exit lb # [ 100.477396] niks3-server[960]: 2026/09/22 11:26:39 INFO lead: released remote=192.168.1.3:49416 worker2 # [ 101.480755] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 101.549366] nix-grpc-daemon[766]: ts=2026-09-22T11:26:40Z level=info event=scheduler_restarting worker1 # [ 101.553609] nix-grpc-daemon[766]: ts=2026-09-22T11:26:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:55828 worker1 # [ 101.553759] nix-grpc-daemon[766]: ts=2026-09-22T11:26:40Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 101.553953] nix-grpc-daemon[766]: ts=2026-09-22T11:26:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:55828 worker1 # [ 101.554069] nix-grpc-daemon[766]: ts=2026-09-22T11:26:40Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 101.554179] nix-grpc-daemon[766]: ts=2026-09-22T11:26:40Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 101.580397] nix-daemon[847]: reaped child process 849, status = succeeded worker2 # [ 101.680835] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.686715] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 101.773472] nix-grpc-daemon[766]: ts=2026-09-22T11:26:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 101.888162] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.893573] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 102.047937] nix-grpc-daemon[766]: E0000 00:00:1790076400.658593 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 102.054724] nix-grpc-daemon[766]: E0000 00:00:1790076400.658880 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 102.094531] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.098175] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 102.299449] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.304777] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 102.564541] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 102.507508] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 102.571947] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 102.511861] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 102.578434] systemd[1]: nix-grpc-daemon.service: Consumed 3.376s CPU time over 1min 23.939s wall clock time, 62.1M memory peak, 74.9K incoming IP traffic, 150.2K outgoing IP traffic. worker1 # [ 102.594325] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 102.597541] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 102.599531] systemd[1]: nix-grpc-daemon.socket: Consumed 615us CPU time over 1min 32.321s wall clock time, 256K memory peak, 104.8K incoming IP traffic, 88.3K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 102.708689] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.714293] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 102.853969] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 102.869275] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 102.896110] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5) worker2 # [ 102.915428] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.920728] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 102.953555] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 192 ms (attempt 2/5) worker2 # [ 103.122073] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.127368] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.147596] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 288 ms (attempt 3/5) worker2 # [ 103.328802] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.334070] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.437851] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 560 ms (attempt 4/5) worker2 # [ 103.537067] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.541285] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.738136] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.743448] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.944657] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.949881] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.001919] nix-grpc-daemon[1231]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 104.009309] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 104.029968] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 104.051771] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 104.080755] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=niks3_push_started pid=1291 worker2 # [ 104.152708] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.155435] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.245040] nix-grpc-daemon[1291]: time=2026-09-22T11:26:42.859Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 104.358045] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.359812] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.365479] nix-daemon[836]: accepted connection from pid 1304, user nix-grpc-daemon (trusted) worker2 # [ 104.388429] nix-daemon[836]: reaped child process 1307, status = succeeded lb # [ 104.385846] niks3-server[960]: 2026/09/22 11:26:42 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 104.397871] niks3-server[960]: 2026/09/22 11:26:43 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 104.411158] nix-grpc-daemon[1291]: time=2026-09-22T11:26:43.024Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 104.414095] nix-grpc-daemon[1291]: time=2026-09-22T11:26:43.024Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" worker2 # [ 104.415837] nix-grpc-daemon[1291]: time=2026-09-22T11:26:43.024Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 104.427341] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=scheduler_take_over lb # [ 104.426143] niks3-server[960]: 2026/09/22 11:26:43 INFO lead: acquired remote=192.168.1.4:40834 lb # [ 104.470158] niks3-server[960]: 2026/09/22 11:26:43 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 104.480360] niks3-server[960]: 2026/09/22 11:26:43 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 104.489196] niks3-server[960]: 2026/09/22 11:26:43 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 104.496078] niks3-server[960]: 2026/09/22 11:26:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 104.505320] niks3-server[960]: 2026/09/22 11:26:43 INFO Signed narinfos id=29 count=1 worker2 # [ 104.507972] nix-grpc-daemon[1291]: time=2026-09-22T11:26:43.120Z level=INFO msg="Uploading 3 narinfos" lb # [ 104.509631] niks3-server[960]: 2026/09/22 11:26:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 104.515081] niks3-server[960]: 2026/09/22 11:26:43 INFO Signed narinfos id=30 count=2 lb # [ 104.518875] niks3-server[960]: 2026/09/22 11:26:43 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 104.537161] niks3-server[960]: 2026/09/22 11:26:43 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 104.547745] niks3-server[960]: 2026/09/22 11:26:43 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 104.554405] niks3-server[960]: 2026/09/22 11:26:43 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo worker2 # [ 104.562401] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=scheduler_connected addr=lb:50051 lb # [ 104.564787] niks3-server[960]: 2026/09/22 11:26:43 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 104.579666] niks3-server[960]: 2026/09/22 11:26:43 INFO Completed upload id=29 worker2 # [ 104.581622] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=oidc_jwks provider=mock keys=1 lb # [ 104.587485] niks3-server[960]: 2026/09/22 11:26:43 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 104.590491] niks3-server[960]: 2026/09/22 11:26:43 INFO Completed upload id=30 worker2 # [ 104.588350] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:58178 worker2 # [ 104.600664] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 104.615998] nix-grpc-daemon[1291]: time=2026-09-22T11:26:43.208Z level=INFO msg="Upload complete. (345ms)" worker2 # [ 104.619371] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 104.624492] nix-daemon[836]: reaped child process 1289, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.02 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 104.811325] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 104.821153] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 104.980510] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 104.983485] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 104.987703] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 104.995123] nix-grpc-daemon[1217]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 105.012165] nix-grpc-daemon[1217]: E0000 00:00:1790076403.603687 1217 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 105.014486] nix-grpc-daemon[1217]: E0000 00:00:1790076403.606051 1217 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 105.044192] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 105.046379] nix-grpc-daemon[1217]: E0000 00:00:1790076403.616781 1259 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 105.049075] nix-grpc-daemon[1217]: E0000 00:00:1790076403.638343 1259 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.051300] nix-grpc-daemon[1217]: E0000 00:00:1790076403.638537 1259 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 105.053767] nix-grpc-daemon[1217]: E0000 00:00:1790076403.654109 1259 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.059288] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 105.062490] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.002440] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:58178 worker2 # [ 105.009119] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.87 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb # [ 106.248685] systemd[1]: Stopping niks3 server... lb # [ 106.249474] niks3-server[960]: 2026/09/22 11:26:44 INFO Shutdown signal received, draining in-flight requests timeout=10s worker1 # [ 106.312514] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:44Z level=info event=scheduler_lock_lost status=200 worker2 # [ 106.260271] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:44Z level=info event=scheduler_lock_lost status=200 lb # [ 106.263064] niks3-server[960]: 2026/09/22 11:26:44 INFO lead: released remote=192.168.1.4:40834 lb # [ 106.277148] systemd[1]: niks3.service: Deactivated successfully. lb # [ 106.278753] systemd[1]: Stopped niks3 server. lb # [ 106.279615] systemd[1]: niks3.service: Consumed 605ms CPU time over 1min 25.059s wall clock time, 26.1M memory peak, 78.5K incoming IP traffic, 114.2K outgoing IP traffic. lb # [ 106.290084] systemd[1]: Starting niks3 server... lb # [ 106.354591] niks3-server[1098]: 2026/09/22 11:26:44 goose: no migrations to run. current version: 20260920000000 lb # [ 106.361285] niks3-server[1098]: 2026/09/22 11:26:44 OK 1_commit_pending_closure.sql (4.77ms) lb # [ 106.364997] niks3-server[1098]: 2026/09/22 11:26:44 OK 2_object_stats_trigger.sql (2.03ms) lb # [ 106.366150] niks3-server[1098]: 2026/09/22 11:26:44 goose: up to current file version: 2 lb # [ 106.367882] niks3-server[1098]: 2026/09/22 11:26:44 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 106.379478] niks3-server[1098]: 2026/09/22 11:26:44 INFO Read proxy enabled — serving cache objects from S3 lb # [ 106.383396] niks3-server[1098]: 2026/09/22 11:26:44 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 106.387792] niks3-server[1098]: 2026/09/22 11:26:44 INFO systemd watchdog enabled interval=15s lb # [ 106.389374] niks3-server[1098]: 2026/09/22 11:26:44 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 106.390792] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 106.883918] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58174 client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 106.895648] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 106.926170] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:45Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 106.933508] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:45Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 106.975166] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 106.991205] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58174 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 107.007279] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 107.022434] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 107.182669] nix-daemon[836]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 107.202189] nix-daemon[836]: reaped child process 1327, status = succeeded lb # [ 107.261562] niks3-server[1098]: 2026/09/22 11:26:45 INFO lead: acquired remote=192.168.1.4:37124 worker2 # [ 107.308320] nix-daemon[836]: accepted connection from pid 1329, user nix-grpc-daemon (trusted) worker2 # [ 107.325865] nix-daemon[836]: reaped child process 1334, status = succeeded lb # [ 107.333231] niks3-server[1098]: 2026/09/22 11:26:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 107.365014] nix-grpc-daemon[1291]: time=2026-09-22T11:26:45.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 107.367304] nix-grpc-daemon[1291]: time=2026-09-22T11:26:45.979Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 107.398773] niks3-server[1098]: 2026/09/22 11:26:45 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 107.409325] niks3-server[1098]: 2026/09/22 11:26:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker2 # [ 107.415365] nix-grpc-daemon[1291]: time=2026-09-22T11:26:46.029Z level=INFO msg="Uploading 1 narinfos" lb # [ 107.418687] niks3-server[1098]: 2026/09/22 11:26:46 INFO Signed narinfos id=31 count=1 lb # [ 107.426688] niks3-server[1098]: 2026/09/22 11:26:46 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 107.439738] niks3-server[1098]: 2026/09/22 11:26:46 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 107.451967] niks3-server[1098]: 2026/09/22 11:26:46 INFO Completed upload id=31 worker2 # [ 107.448733] nix-grpc-daemon[1291]: time=2026-09-22T11:26:46.063Z level=INFO msg="Upload complete. (408ms)" worker2 # [ 107.461346] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 107.480692] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 107.488586] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 107.494801] nix-daemon[836]: reaped child process 1322, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker2 # [ 107.507180] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 108.558601] envoy[908]: conn peer=[2001:db8:1::1]:42444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=30 lb # [ 108.564235] envoy[908]: conn peer=[2001:db8:1::3]:50992 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=9355 tx=2705 ms=81970 lb # [ 108.569431] envoy[908]: conn peer=[2001:db8:1::1]:47402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=282 ms=1763 client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 108.765139] nix-daemon[836]: reaped child process 1338, status = succeeded worker2 # [ 108.888741] nix-daemon[836]: accepted connection from pid 1354, user nix-grpc-daemon (trusted) worker2 # [ 108.908219] nix-daemon[836]: reaped child process 1357, status = succeeded lb # [ 108.914086] niks3-server[1098]: 2026/09/22 11:26:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 108.932463] nix-grpc-daemon[1291]: time=2026-09-22T11:26:47.544Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 108.934248] nix-grpc-daemon[1291]: time=2026-09-22T11:26:47.544Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 108.975657] niks3-server[1098]: 2026/09/22 11:26:47 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 108.985669] niks3-server[1098]: 2026/09/22 11:26:47 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst worker2 # [ 108.996116] nix-grpc-daemon[1291]: time=2026-09-22T11:26:47.610Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.997642] niks3-server[1098]: 2026/09/22 11:26:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign lb # [ 109.002070] niks3-server[1098]: 2026/09/22 11:26:47 INFO Signed narinfos id=32 count=1 lb # [ 109.007317] niks3-server[1098]: 2026/09/22 11:26:47 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 109.014020] niks3-server[1098]: 2026/09/22 11:26:47 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 109.024843] niks3-server[1098]: 2026/09/22 11:26:47 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo lb # [ 109.032675] niks3-server[1098]: 2026/09/22 11:26:47 INFO Completed upload id=32 worker2 # [ 109.034284] nix-grpc-daemon[1291]: time=2026-09-22T11:26:47.648Z level=INFO msg="Upload complete. (313ms)" worker2 # [ 109.057377] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 109.076853] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:47Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 109.081347] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:47Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 109.086300] nix-daemon[836]: reaped child process 1336, status = succeeded worker2 # [ 109.098107] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 109.114037] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 109.131256] nix-grpc-daemon[1291]: time=2026-09-22T11:26:47.745Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 109.141813] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 109.155551] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 109.310258] nix-daemon[836]: accepted connection from pid 1363, user nix-grpc-daemon (trusted) worker2 # [ 109.334999] nix-daemon[836]: reaped child process 1366, status = succeeded worker2 # [ 109.436851] nix-daemon[836]: accepted connection from pid 1368, user nix-grpc-daemon (trusted) worker2 # [ 109.458770] nix-daemon[836]: reaped child process 1371, status = succeeded lb # [ 109.461056] niks3-server[1098]: 2026/09/22 11:26:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 109.471930] nix-grpc-daemon[1291]: time=2026-09-22T11:26:48.086Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 109.473682] nix-grpc-daemon[1291]: time=2026-09-22T11:26:48.086Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 109.490973] niks3-server[1098]: 2026/09/22 11:26:48 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst worker2 # [ 109.507636] nix-grpc-daemon[1291]: time=2026-09-22T11:26:48.122Z level=INFO msg="Uploading 1 narinfos" lb # [ 109.507541] niks3-server[1098]: 2026/09/22 11:26:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign lb # [ 109.519906] niks3-server[1098]: 2026/09/22 11:26:48 INFO Signed narinfos id=33 count=1 lb # [ 109.528882] niks3-server[1098]: 2026/09/22 11:26:48 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls worker2 # [ 109.543490] nix-grpc-daemon[1291]: time=2026-09-22T11:26:48.158Z level=INFO msg="Upload complete. (372ms)" lb # [ 109.542151] niks3-server[1098]: 2026/09/22 11:26:48 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 109.548956] niks3-server[1098]: 2026/09/22 11:26:48 INFO Completed upload id=33 worker2 # [ 109.552312] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 duration_s=0 paths=1 nar_bytes_in=656 lb # [ 109.552545] niks3-server[1098]: 2026/09/22 11:26:48 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 109.567761] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 109.574824] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 109.580673] nix-daemon[836]: reaped child process 1361, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker2 # [ 109.595585] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 110.506080] nix-daemon[836]: reaped child process 1375, status = succeeded worker2 # [ 110.580562] nix-daemon[836]: accepted connection from pid 1391, user nix-grpc-daemon (trusted) worker2 # [ 110.605930] nix-daemon[836]: reaped child process 1394, status = succeeded lb # [ 110.603099] niks3-server[1098]: 2026/09/22 11:26:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 110.620154] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.233Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 110.622176] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.233Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 110.649101] niks3-server[1098]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 110.658203] niks3-server[1098]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 110.669135] niks3-server[1098]: 2026/09/22 11:26:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign worker2 # [ 110.675317] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.289Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.676221] niks3-server[1098]: 2026/09/22 11:26:49 INFO Signed narinfos id=34 count=1 lb # [ 110.684344] niks3-server[1098]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 110.694303] niks3-server[1098]: 2026/09/22 11:26:49 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 110.701354] niks3-server[1098]: 2026/09/22 11:26:49 INFO Completed upload id=34 worker2 # [ 110.705853] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.320Z level=INFO msg="Upload complete. (298ms)" lb # [ 110.707689] niks3-server[1098]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker2 # [ 110.724946] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 110.740480] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 110.747564] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 110.752222] nix-daemon[836]: reaped child process 1373, status = succeeded worker2 # [ 110.770237] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 110.786642] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 110.802120] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.416Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 110.813061] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 110.826849] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 110.978440] nix-daemon[836]: accepted connection from pid 1400, user nix-grpc-daemon (trusted) worker2 # [ 111.002195] nix-daemon[836]: reaped child process 1403, status = succeeded worker2 # [ 111.106359] nix-daemon[836]: accepted connection from pid 1405, user nix-grpc-daemon (trusted) worker2 # [ 111.128420] nix-daemon[836]: reaped child process 1408, status = succeeded lb # [ 111.124351] niks3-server[1098]: 2026/09/22 11:26:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.134923] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.749Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 111.136637] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.749Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 111.153710] niks3-server[1098]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker2 # [ 111.168209] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.782Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.168630] niks3-server[1098]: 2026/09/22 11:26:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 111.182038] niks3-server[1098]: 2026/09/22 11:26:49 INFO Signed narinfos id=35 count=1 lb # [ 111.189529] niks3-server[1098]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 111.201250] niks3-server[1098]: 2026/09/22 11:26:49 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete worker2 # [ 111.209526] nix-grpc-daemon[1291]: time=2026-09-22T11:26:49.824Z level=INFO msg="Upload complete. (366ms)" lb # [ 111.209498] niks3-server[1098]: 2026/09/22 11:26:49 INFO Completed upload id=35 lb # [ 111.211924] niks3-server[1098]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 111.221131] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58174 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 111.232305] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 worker2 # [ 111.239518] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 111.245152] nix-daemon[836]: reaped child process 1398, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 111.259639] nix-daemon[836]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 112.518163] nix-daemon[836]: reaped child process 1412, status = succeeded worker2 # [ 112.624836] nix-daemon[836]: accepted connection from pid 1428, user nix-grpc-daemon (trusted) worker2 # [ 112.648738] nix-daemon[836]: reaped child process 1431, status = succeeded lb # [ 112.646893] niks3-server[1098]: 2026/09/22 11:26:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 112.672339] nix-grpc-daemon[1291]: time=2026-09-22T11:26:51.284Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 112.674321] nix-grpc-daemon[1291]: time=2026-09-22T11:26:51.284Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 112.691110] niks3-server[1098]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 112.708837] niks3-server[1098]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 112.722105] nix-grpc-daemon[1291]: time=2026-09-22T11:26:51.336Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.720705] niks3-server[1098]: 2026/09/22 11:26:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 112.732146] niks3-server[1098]: 2026/09/22 11:26:51 INFO Signed narinfos id=36 count=1 lb # [ 112.736704] niks3-server[1098]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 112.747132] niks3-server[1098]: 2026/09/22 11:26:51 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 112.756297] niks3-server[1098]: 2026/09/22 11:26:51 INFO Completed upload id=36 lb # [ 112.759000] niks3-server[1098]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 112.756436] nix-grpc-daemon[1291]: time=2026-09-22T11:26:51.371Z level=INFO msg="Upload complete. (286ms)" worker2 # [ 112.773648] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58174 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 112.790032] nix-daemon[836]: reaped child process 1410, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.16 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.17 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 6.86 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 113.237635] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 113.426295] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:52Z level=info event=signal_exit lb # [ 114.273800] niks3-server[1098]: 2026/09/22 11:26:52 INFO lead: released remote=192.168.1.4:37124 worker1 # [ 115.332596] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:53Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 115.275125] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:53Z level=info event=scheduler_restarting worker2 # [ 115.280231] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:53Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 115.284668] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:58178 worker2 # [ 115.289697] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:53Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 115.294250] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:58178 worker2 # [ 115.299465] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:53Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 115.323161] nix-daemon[836]: reaped child process 1283, status = succeeded worker1 # [ 115.532315] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.537829] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 115.490776] nix-grpc-daemon[1231]: ts=2026-09-22T11:26:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 115.497512] nix-grpc-daemon[1231]: E0000 00:00:1790076414.111632 1276 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker2 # [ 115.504166] nix-grpc-daemon[1231]: E0000 00:00:1790076414.111792 1276 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 115.738617] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.744166] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 115.945172] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.950676] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.151520] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.157092] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.359783] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.364139] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.560851] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.566211] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 116.552908] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 116.559727] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 116.564643] systemd[1]: nix-grpc-daemon.service: Consumed 1.732s CPU time over 17.303s wall clock time, 38.3M memory peak, 48.8K incoming IP traffic, 91.4K outgoing IP traffic. worker2 # [ 116.578452] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 116.586328] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 116.587979] systemd[1]: nix-grpc-daemon.socket: Consumed 738us CPU time over 17.569s wall clock time, 256K memory peak, 39.2K incoming IP traffic, 29.4K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 116.766634] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.772762] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1 # [ 116.973850] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.979156] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 117.179943] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.185089] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 117.318202] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_take_over lb # [ 117.256390] niks3-server[1098]: 2026/09/22 11:26:55 INFO lead: acquired remote=192.168.1.3:59890 worker1 # [ 117.387628] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.391997] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 117.588670] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.593998] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 117.794970] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.808859] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:36370 worker1 # [ 117.819370] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 worker1 # [ 118.101676] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 118.117064] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 118.141639] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 118.158266] nix-grpc-daemon[1217]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5) worker1 # [ 118.170430] nix-grpc-daemon[1217]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 200 ms (attempt 2/5) worker1 # [ 118.379899] nix-grpc-daemon[1217]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 3/5) worker1 # [ 118.497919] nix-grpc-daemon[1217]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 440 ms (attempt 4/5) lb # [ 118.560491] envoy[908]: conn peer=[2001:db8:1::1]:47418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9766 tx=4079 ms=5904 lb # [ 118.560824] envoy[908]: conn peer=[2001:db8:1::4]:37980 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=13149 tx=995 ms=16003 worker1 # [ 118.941916] nix-grpc-daemon[1217]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 118.947188] nix-grpc-daemon[1217]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 118.979745] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:57Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 118.986882] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:57Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 119.027614] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 119.043295] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36384 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 119.058019] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 119.072742] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 119.101482] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:57Z level=info event=niks3_push_started pid=1293 worker1 # [ 119.250986] nix-grpc-daemon[1293]: time=2026-09-22T11:26:57.861Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 119.361968] nix-daemon[847]: accepted connection from pid 1306, user nix-grpc-daemon (trusted) worker1 # [ 119.384731] nix-daemon[847]: reaped child process 1309, status = succeeded worker1 # [ 119.484392] nix-daemon[847]: accepted connection from pid 1311, user nix-grpc-daemon (trusted) worker1 # [ 119.501190] nix-daemon[847]: reaped child process 1314, status = succeeded lb # [ 119.447064] niks3-server[1098]: 2026/09/22 11:26:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.520080] nix-grpc-daemon[1293]: time=2026-09-22T11:26:58.130Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 119.521573] nix-grpc-daemon[1293]: time=2026-09-22T11:26:58.130Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 119.482334] niks3-server[1098]: 2026/09/22 11:26:58 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst worker1 # [ 119.565595] nix-grpc-daemon[1293]: time=2026-09-22T11:26:58.176Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.495010] niks3-server[1098]: 2026/09/22 11:26:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 119.509819] niks3-server[1098]: 2026/09/22 11:26:58 INFO Signed narinfos id=37 count=1 lb # [ 119.535403] niks3-server[1098]: 2026/09/22 11:26:58 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete worker1 # [ 119.606667] nix-grpc-daemon[1293]: time=2026-09-22T11:26:58.217Z level=INFO msg="Upload complete. (351ms)" lb # [ 119.549483] niks3-server[1098]: 2026/09/22 11:26:58 INFO Completed upload id=37 worker1 # [ 119.616986] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 119.634283] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 119.643462] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 119.646145] nix-daemon[847]: reaped child process 1291, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker1 # [ 119.658107] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 120.518691] nix-daemon[847]: reaped child process 1318, status = succeeded worker1 # [ 120.656220] nix-daemon[847]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker1 # [ 120.676579] nix-daemon[847]: reaped child process 1337, status = succeeded lb # [ 120.617782] niks3-server[1098]: 2026/09/22 11:26:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 120.700206] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 120.703934] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.307Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 120.674243] niks3-server[1098]: 2026/09/22 11:26:59 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 120.686691] niks3-server[1098]: 2026/09/22 11:26:59 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 120.696753] niks3-server[1098]: 2026/09/22 11:26:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign worker1 # [ 120.759358] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.368Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.706089] niks3-server[1098]: 2026/09/22 11:26:59 INFO Signed narinfos id=38 count=1 lb # [ 120.717715] niks3-server[1098]: 2026/09/22 11:26:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 120.727358] niks3-server[1098]: 2026/09/22 11:26:59 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 120.734830] niks3-server[1098]: 2026/09/22 11:26:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo lb # [ 120.739461] niks3-server[1098]: 2026/09/22 11:26:59 INFO Completed upload id=38 worker1 # [ 120.802038] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.413Z level=INFO msg="Upload complete. (331ms)" worker1 # [ 120.823884] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 120.840415] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 120.847923] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 120.853695] nix-daemon[847]: reaped child process 1316, status = succeeded worker1 # [ 120.866623] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 120.880827] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 120.893485] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.504Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 120.902677] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 120.915996] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 121.062371] nix-daemon[847]: accepted connection from pid 1343, user nix-grpc-daemon (trusted) worker1 # [ 121.085456] nix-daemon[847]: reaped child process 1346, status = succeeded worker1 # [ 121.183738] nix-daemon[847]: accepted connection from pid 1348, user nix-grpc-daemon (trusted) worker1 # [ 121.200775] nix-daemon[847]: reaped child process 1351, status = succeeded lb # [ 121.143735] niks3-server[1098]: 2026/09/22 11:26:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 121.214198] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.824Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 121.215685] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.824Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 121.164132] niks3-server[1098]: 2026/09/22 11:26:59 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker1 # [ 121.239618] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.850Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.171977] niks3-server[1098]: 2026/09/22 11:26:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign lb # [ 121.181993] niks3-server[1098]: 2026/09/22 11:26:59 INFO Signed narinfos id=39 count=1 lb # [ 121.186223] niks3-server[1098]: 2026/09/22 11:26:59 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 121.196629] niks3-server[1098]: 2026/09/22 11:26:59 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 121.203648] niks3-server[1098]: 2026/09/22 11:26:59 INFO Completed upload id=39 worker1 # [ 121.266384] nix-grpc-daemon[1293]: time=2026-09-22T11:26:59.877Z level=INFO msg="Upload complete. (335ms)" lb # [ 121.206294] niks3-server[1098]: 2026/09/22 11:26:59 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker1 # [ 121.272235] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 121.283853] nix-grpc-daemon[1217]: ts=2026-09-22T11:26:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 121.289653] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 121.294168] nix-daemon[847]: reaped child process 1341, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... worker1 # [ 121.306373] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 122.166969] nix-daemon[847]: reaped child process 1355, status = succeeded worker1 # [ 122.286211] nix-daemon[847]: accepted connection from pid 1371, user nix-grpc-daemon (trusted) worker1 # [ 122.307531] nix-daemon[847]: reaped child process 1374, status = succeeded lb # [ 122.249704] niks3-server[1098]: 2026/09/22 11:27:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 122.323492] nix-grpc-daemon[1293]: time=2026-09-22T11:27:00.933Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 122.325261] nix-grpc-daemon[1293]: time=2026-09-22T11:27:00.933Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 122.279833] niks3-server[1098]: 2026/09/22 11:27:00 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 122.297264] niks3-server[1098]: 2026/09/22 11:27:00 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 122.311003] niks3-server[1098]: 2026/09/22 11:27:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker1 # [ 122.379467] nix-grpc-daemon[1293]: time=2026-09-22T11:27:00.990Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.320981] niks3-server[1098]: 2026/09/22 11:27:00 INFO Signed narinfos id=40 count=1 lb # [ 122.323434] niks3-server[1098]: 2026/09/22 11:27:00 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 122.338085] niks3-server[1098]: 2026/09/22 11:27:00 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 122.345137] niks3-server[1098]: 2026/09/22 11:27:00 INFO Completed upload id=40 worker1 # [ 122.407086] nix-grpc-daemon[1293]: time=2026-09-22T11:27:01.018Z level=INFO msg="Upload complete. (290ms)" lb # [ 122.348402] niks3-server[1098]: 2026/09/22 11:27:00 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker1 # [ 122.425445] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 122.442831] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 122.447220] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 122.452132] nix-daemon[847]: reaped child process 1353, status = succeeded worker1 # [ 122.471237] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 122.487059] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36384 client # copying 1 paths to 'lb:50051'... worker1 # [ 122.501761] nix-grpc-daemon[1293]: time=2026-09-22T11:27:01.112Z level=INFO msg="All 2 paths already cached" client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 122.508104] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 122.521398] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 122.677166] nix-daemon[847]: accepted connection from pid 1380, user nix-grpc-daemon (trusted) worker1 # [ 122.697063] nix-daemon[847]: reaped child process 1383, status = succeeded worker1 # [ 122.797769] nix-daemon[847]: accepted connection from pid 1385, user nix-grpc-daemon (trusted) worker1 # [ 122.815813] nix-daemon[847]: reaped child process 1388, status = succeeded lb # [ 122.758424] niks3-server[1098]: 2026/09/22 11:27:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 122.830377] nix-grpc-daemon[1293]: time=2026-09-22T11:27:01.441Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 122.831892] nix-grpc-daemon[1293]: time=2026-09-22T11:27:01.441Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 122.785668] niks3-server[1098]: 2026/09/22 11:27:01 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst worker1 # [ 122.861788] nix-grpc-daemon[1293]: time=2026-09-22T11:27:01.472Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.801282] niks3-server[1098]: 2026/09/22 11:27:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 122.813233] niks3-server[1098]: 2026/09/22 11:27:01 INFO Signed narinfos id=41 count=1 lb # [ 122.815971] niks3-server[1098]: 2026/09/22 11:27:01 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls worker1 # [ 122.892493] nix-grpc-daemon[1293]: time=2026-09-22T11:27:01.503Z level=INFO msg="Upload complete. (355ms)" lb # [ 122.830545] niks3-server[1098]: 2026/09/22 11:27:01 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 122.838534] niks3-server[1098]: 2026/09/22 11:27:01 INFO Completed upload id=41 lb # [ 122.841241] niks3-server[1098]: 2026/09/22 11:27:01 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 122.900124] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 122.908738] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 122.918498] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 122.922780] nix-daemon[847]: reaped child process 1378, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker1 # [ 122.934654] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 124.216569] nix-daemon[847]: reaped child process 1392, status = succeeded worker1 # [ 124.307549] nix-daemon[847]: accepted connection from pid 1408, user nix-grpc-daemon (trusted) worker1 # [ 124.330352] nix-daemon[847]: reaped child process 1411, status = succeeded lb # [ 124.274103] niks3-server[1098]: 2026/09/22 11:27:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 124.346448] nix-grpc-daemon[1293]: time=2026-09-22T11:27:02.955Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 124.348165] nix-grpc-daemon[1293]: time=2026-09-22T11:27:02.955Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 124.310210] niks3-server[1098]: 2026/09/22 11:27:02 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 124.323117] niks3-server[1098]: 2026/09/22 11:27:02 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker1 # [ 124.397679] nix-grpc-daemon[1293]: time=2026-09-22T11:27:03.008Z level=INFO msg="Uploading 1 narinfos" lb # [ 124.337979] niks3-server[1098]: 2026/09/22 11:27:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 124.348220] niks3-server[1098]: 2026/09/22 11:27:02 INFO Signed narinfos id=42 count=1 lb # [ 124.351340] niks3-server[1098]: 2026/09/22 11:27:02 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 124.364109] niks3-server[1098]: 2026/09/22 11:27:02 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 124.368533] niks3-server[1098]: 2026/09/22 11:27:02 INFO Completed upload id=42 worker1 # [ 124.430250] nix-grpc-daemon[1293]: time=2026-09-22T11:27:03.041Z level=INFO msg="Upload complete. (285ms)" lb # [ 124.370372] niks3-server[1098]: 2026/09/22 11:27:02 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 124.445284] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 124.463374] nix-daemon[847]: reaped child process 1390, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.62 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds) worker2 # [ 124.694796] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 124.702374] systemd[1]: Starting scheduler-token.service... worker2 # [ 124.774162] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 124.778541] systemd[1]: Finished scheduler-token.service. worker2 # [ 124.783748] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 74ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 124.789870] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 124.910753] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 124.913142] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 124.917083] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 124.922548] nix-grpc-daemon[1452]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 124.944514] nix-grpc-daemon[1452]: E0000 00:00:1790076423.536553 1452 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.946991] nix-grpc-daemon[1452]: E0000 00:00:1790076423.540357 1452 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 124.972868] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 124.975075] nix-grpc-daemon[1452]: E0000 00:00:1790076423.548500 1494 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.977788] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 124.978585] nix-grpc-daemon[1452]: E0000 00:00:1790076423.570760 1494 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 124.980965] nix-grpc-daemon[1452]: E0000 00:00:1790076423.570968 1494 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.983271] nix-grpc-daemon[1452]: E0000 00:00:1790076423.586608 1494 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 124.992268] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 125.064172] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:03Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 125.069635] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:36370 worker1 # [ 125.075162] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.30 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 125.508738] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 64a8ae1093074b8f9728fc78da4c2419 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.10 seconds) worker1 # [ 125.639831] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36384 client # [ 125.730631] nix[1361]: don't know how to build these paths: client # [ 125.731958] nix[1361]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 125.661572] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 125.669217] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 125.707449] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 125.724991] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36384 client # [ 125.824388] nix[1361]: copying 1 paths to 'lb:50051'... worker1 # [ 125.741273] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 client # [ 125.826845] nix[1361]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 125.757181] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 125.920531] nix-daemon[847]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker1 # [ 125.949880] nix-daemon[847]: reaped child process 1427, status = succeeded worker1 # [ 126.056156] nix-daemon[847]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) worker1 # [ 126.078667] nix-daemon[847]: reaped child process 1432, status = succeeded lb # [ 126.015063] niks3-server[1098]: 2026/09/22 11:27:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 126.088422] nix-grpc-daemon[1293]: time=2026-09-22T11:27:04.698Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 126.089893] nix-grpc-daemon[1293]: time=2026-09-22T11:27:04.698Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 126.047920] niks3-server[1098]: 2026/09/22 11:27:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker1 # [ 126.118928] nix-grpc-daemon[1293]: time=2026-09-22T11:27:04.729Z level=INFO msg="Uploading 1 narinfos" lb # [ 126.062682] niks3-server[1098]: 2026/09/22 11:27:04 INFO Signed narinfos id=43 count=1 lb # [ 126.072292] niks3-server[1098]: 2026/09/22 11:27:04 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 126.082876] niks3-server[1098]: 2026/09/22 11:27:04 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker1 # [ 126.158767] nix-grpc-daemon[1293]: time=2026-09-22T11:27:04.769Z level=INFO msg="Upload complete. (383ms)" lb # [ 126.097722] niks3-server[1098]: 2026/09/22 11:27:04 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete worker1 # [ 126.168276] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 126.109769] niks3-server[1098]: 2026/09/22 11:27:04 INFO Completed upload id=43 lb # [ 126.113588] niks3-server[1098]: 2026/09/22 11:27:04 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker1 # [ 126.183597] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 worker1 # [ 126.189993] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 126.195035] nix-daemon[847]: reaped child process 1422, status = succeeded client # [ 126.287944] nix[1361]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker1 # [ 126.208196] nix-daemon[847]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 128.234214] systemd-journald[269]: Received client request to rotate journal, rotating. worker2 # [ 128.255106] systemd-journald[269]: Vacuuming done, freed 0B of archived journals from /var/log/journal/96eee89c02c34bcca998924ebf14576d. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 128.364180] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds) client: must succeed: sleep 5.0 lb # [ 128.558946] envoy[908]: conn peer=[2001:db8:1::1]:60720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=4102 ms=6356 worker1 # [ 129.010344] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:07Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1 # [ 133.637463] nix-daemon[847]: reaped child process 1436, status = succeeded worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 133.736828] nix-daemon[847]: accepted connection from pid 1477, user nix-grpc-daemon (trusted) worker1 # [ 133.760652] nix-daemon[847]: reaped child process 1480, status = succeeded lb # [ 133.697496] niks3-server[1098]: 2026/09/22 11:27:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 133.772399] nix-grpc-daemon[1293]: time=2026-09-22T11:27:12.380Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 133.773865] nix-grpc-daemon[1293]: time=2026-09-22T11:27:12.380Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 133.738822] niks3-server[1098]: 2026/09/22 11:27:12 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 133.824591] nix-grpc-daemon[1293]: time=2026-09-22T11:27:12.435Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.751529] niks3-server[1098]: 2026/09/22 11:27:12 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 133.765861] niks3-server[1098]: 2026/09/22 11:27:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 133.772338] niks3-server[1098]: 2026/09/22 11:27:12 INFO Signed narinfos id=44 count=1 lb # [ 133.774278] niks3-server[1098]: 2026/09/22 11:27:12 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 133.784608] niks3-server[1098]: 2026/09/22 11:27:12 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 133.789894] niks3-server[1098]: 2026/09/22 11:27:12 INFO Completed upload id=44 worker1 # [ 133.852616] nix-grpc-daemon[1293]: time=2026-09-22T11:27:12.463Z level=INFO msg="Upload complete. (229ms)" lb # [ 133.792628] niks3-server[1098]: 2026/09/22 11:27:12 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker1 # [ 133.864851] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36384 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 133.879425] nix-daemon[847]: reaped child process 1434, status = succeeded client # [ 133.978709] systemd[1]: rs.service: Deactivated successfully. client # [ 133.983956] systemd[1]: rs.service: Consumed 199ms CPU time over 8.471s wall clock time, 39.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 134.011905] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:12Z level=info event=signal_exit lb # [ 134.260232] niks3-server[1098]: 2026/09/22 11:27:12 INFO lead: released remote=192.168.1.3:59890 lb # [ 134.933713] niks3-server[1098]: 2026/09/22 11:27:13 INFO lead: acquired remote=192.168.1.4:34514 worker2 # [ 134.935452] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:13Z level=info event=scheduler_take_over worker2 # [ 135.261920] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:13Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 135.328421] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:13Z level=info event=scheduler_restarting worker1 # [ 135.328834] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:13Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 135.328975] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:36370 worker1 # [ 135.329088] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:13Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 135.329198] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:36370 worker1 # [ 135.329305] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:13Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 135.377597] nix-daemon[847]: reaped child process 1283, status = succeeded worker2 # [ 135.463534] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 135.469223] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 135.545977] nix-grpc-daemon[1217]: ts=2026-09-22T11:27:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 135.648159] nix-grpc-daemon[1217]: E0000 00:00:1790076434.258771 1262 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 135.654744] nix-grpc-daemon[1217]: E0000 00:00:1790076434.259086 1262 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 135.670199] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 135.687564] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:14Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 135.692250] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:14Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:38464 worker2 # [ 135.697586] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:14Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.17 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 137.214690] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 137.219703] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 137.224962] systemd[1]: nix-grpc-daemon.service: Consumed 1.986s CPU time over 32.392s wall clock time, 39.1M memory peak, 48.1K incoming IP traffic, 89K outgoing IP traffic. worker1 # [ 137.245702] systemd[1]: Starting Nix worker-protocol over gRPC... worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 137.398453] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 137.400757] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 137.404550] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 137.409798] nix-grpc-daemon[1487]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 137.411259] nix-grpc-daemon[1487]: E0000 00:00:1790076436.020092 1487 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 137.451770] nix-grpc-daemon[1487]: E0000 00:00:1790076436.026163 1487 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 137.455286] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 137.457550] nix-grpc-daemon[1487]: E0000 00:00:1790076436.032094 1529 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 137.460113] nix-grpc-daemon[1487]: E0000 00:00:1790076436.047885 1529 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 137.465473] nix-grpc-daemon[1487]: E0000 00:00:1790076436.048049 1529 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 137.469806] nix-grpc-daemon[1487]: E0000 00:00:1790076436.062586 1529 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 137.472304] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 137.473146] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 137.411342] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:38464 worker2 # [ 137.416675] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.03 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 138.528810] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 5a19b1e193fa430abc3037b476210543 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds) lb # [ 138.559162] envoy[908]: conn peer=[2001:db8:1::1]:57544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1428 ms=8240 worker2 # [ 138.595814] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38470 worker2 # [ 138.611883] nix-daemon[836]: accepted connection from pid 1452, user nix-grpc-daemon (trusted) client # [ 138.774599] nix[1397]: don't know how to build these paths: client # [ 138.776357] nix[1397]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 138.644157] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:17Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 138.650113] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:17Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 138.699154] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38470 worker2 # [ 138.715446] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5) worker2 # [ 138.737103] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 188 ms (attempt 2/5) worker2 # [ 138.927314] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 212 ms (attempt 3/5) worker2 # [ 139.141367] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 4/5) lb # [ 138.564453] envoy[908]: conn peer=[2001:db8:1::3]:36150 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2953 tx=971 ms=30479 worker2 # [ 139.232803] nix-grpc-daemon[1452]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 139.239650] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 139.257965] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38470 client # [ 139.416257] nix[1397]: copying 1 paths to 'lb:50051'... client # [ 139.421865] nix[1397]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker2 # [ 139.278023] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38470 worker2 # [ 139.292380] nix-daemon[836]: accepted connection from pid 1452, user nix-grpc-daemon (trusted) worker2 # [ 139.324806] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:17Z level=info event=niks3_push_started pid=1577 worker2 # [ 139.471112] nix-grpc-daemon[1577]: time=2026-09-22T11:27:18.085Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 139.580205] nix-daemon[836]: accepted connection from pid 1589, user nix-grpc-daemon (trusted) worker2 # [ 139.598027] nix-daemon[836]: reaped child process 1592, status = succeeded worker2 # [ 139.731067] nix-daemon[836]: accepted connection from pid 1594, user nix-grpc-daemon (trusted) worker2 # [ 139.754933] nix-daemon[836]: reaped child process 1601, status = succeeded lb # [ 139.751797] niks3-server[1098]: 2026/09/22 11:27:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 139.763143] nix-grpc-daemon[1577]: time=2026-09-22T11:27:18.377Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 139.766434] nix-grpc-daemon[1577]: time=2026-09-22T11:27:18.379Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 139.786609] niks3-server[1098]: 2026/09/22 11:27:18 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst worker2 # [ 139.801000] nix-grpc-daemon[1577]: time=2026-09-22T11:27:18.415Z level=INFO msg="Uploading 1 narinfos" lb # [ 139.800677] niks3-server[1098]: 2026/09/22 11:27:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign lb # [ 139.810922] niks3-server[1098]: 2026/09/22 11:27:18 INFO Signed narinfos id=45 count=1 lb # [ 139.817289] niks3-server[1098]: 2026/09/22 11:27:18 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 139.829797] niks3-server[1098]: 2026/09/22 11:27:18 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 139.834444] niks3-server[1098]: 2026/09/22 11:27:18 INFO Completed upload id=45 worker2 # [ 139.830875] nix-grpc-daemon[1577]: time=2026-09-22T11:27:18.445Z level=INFO msg="Upload complete. (356ms)" lb # [ 139.839447] niks3-server[1098]: 2026/09/22 11:27:18 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker2 # [ 139.841337] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38470 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 139.850282] nix-grpc-daemon[1452]: ts=2026-09-22T11:27:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38470 worker2 # [ 139.858824] nix-daemon[836]: accepted connection from pid 1452, user nix-grpc-daemon (trusted) worker2 # [ 139.860910] nix-daemon[836]: reaped child process 1575, status = succeeded worker2 # [ 139.872629] nix-daemon[836]: accepted connection from pid 1452, user nix-grpc-daemon (trusted) client # [ 140.015095] nix[1397]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 142.260554] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1452 (nix-grpc-daemon) on client request. worker2 # [ 142.262137] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 142.273148] niks3-server[1098]: 2026/09/22 11:27:20 INFO lead: released remote=192.168.1.4:34514 worker2 # [ 142.280319] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 142.281851] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. client # [ 142.422507] nix[1397]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker2 # [ 142.283105] systemd[1]: nix-grpc-daemon.service: Consumed 602ms CPU time over 17.484s wall clock time, 34.6M memory peak, 15.7K incoming IP traffic, 27.7K outgoing IP traffic. worker1 # [ 142.339471] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:20Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" client # [ 142.428871] nix[1397]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 142.304249] systemd[1]: Starting scheduler-token.service... worker2 # [ 142.326783] nix-daemon[836]: reaped child process 1567, status = succeeded worker2 # [ 142.333375] nix-daemon[836]: reaped child process 1603, status = succeeded worker2 # [ 142.340423] nix-daemon[1605]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 142.348576] nix-daemon[836]: reaped child process 1605, status = failed with exit code 1 lb # [ 142.360600] niks3-server[1098]: 2026/09/22 11:27:20 INFO lead: acquired remote=192.168.1.3:38462 worker1 # [ 142.427378] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_take_over worker2 # [ 142.377179] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1. worker2 # [ 142.379414] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 142.382902] systemd[1]: Finished scheduler-token.service. worker2 # [ 142.383671] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 142.386852] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 142.539515] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 142.484443] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 142.486575] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 142.490160] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 142.495448] nix-grpc-daemon[1645]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 142.577726] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 142.520156] nix-grpc-daemon[1645]: E0000 00:00:1790076441.109371 1645 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 142.522552] nix-grpc-daemon[1645]: E0000 00:00:1790076441.112983 1645 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.32 seconds) client: must succeed: sleep 5.0 worker2 # [ 142.547768] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 142.550003] nix-grpc-daemon[1645]: E0000 00:00:1790076441.122303 1684 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 142.552594] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 142.553422] nix-grpc-daemon[1645]: E0000 00:00:1790076441.145987 1684 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 142.558517] nix-grpc-daemon[1645]: E0000 00:00:1790076441.146132 1684 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 142.562939] nix-grpc-daemon[1645]: E0000 00:00:1790076441.161852 1684 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 142.565992] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 142.567779] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 142.778740] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.783854] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 142.769083] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 142.774446] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client # [ 142.925621] nix[1397]: scheduler at lb:50051: no healthy upstream, reconnecting worker1 # [ 142.984802] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 142.990331] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 142.975363] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 142.980709] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.191164] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.196946] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 143.181791] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 143.186956] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.399688] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.404377] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 143.388072] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 143.393139] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.600606] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.606065] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 143.595908] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 143.600317] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.806827] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.812404] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client # [ 143.929392] nix[1397]: scheduler at lb:50051: no healthy upstream, reconnecting worker2 # [ 143.796836] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 143.802006] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 144.013283] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.018729] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 144.003143] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 144.008386] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 144.219479] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.224998] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 144.209286] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.295434] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 144.299882] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:46092 worker1 # [ 144.305156] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 144.428563] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.432936] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:46092 worker1 # [ 144.437806] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 145.857325] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:24Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 145.863362] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:24Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 145.855863] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49346 worker2 # [ 145.870835] nix-daemon[836]: accepted connection from pid 1645, user nix-grpc-daemon (trusted) worker2 # [ 145.889527] nix-daemon[836]: accepted connection from pid 1645, user nix-grpc-daemon (trusted) client # [ 146.052551] nix[1397]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker2 # [ 145.912124] nix-daemon[836]: accepted connection from pid 1645, user nix-grpc-daemon (trusted) client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker2 # [ 147.930998] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:26Z level=info event=niks3_push_started pid=1726 worker2 # [ 147.959708] nix-daemon[836]: reaped child process 1701, status = succeeded worker2 # [ 148.066509] nix-grpc-daemon[1726]: time=2026-09-22T11:27:26.680Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 148.172206] nix-daemon[836]: accepted connection from pid 1739, user nix-grpc-daemon (trusted) worker2 # [ 148.191502] nix-daemon[836]: reaped child process 1742, status = succeeded lb # [ 148.197228] niks3-server[1098]: 2026/09/22 11:27:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 148.209885] nix-grpc-daemon[1726]: time=2026-09-22T11:27:26.822Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 148.211339] nix-grpc-daemon[1726]: time=2026-09-22T11:27:26.822Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 148.239691] niks3-server[1098]: 2026/09/22 11:27:26 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 148.252007] niks3-server[1098]: 2026/09/22 11:27:26 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 148.264551] niks3-server[1098]: 2026/09/22 11:27:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 148.281266] niks3-server[1098]: 2026/09/22 11:27:26 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 148.289969] niks3-server[1098]: 2026/09/22 11:27:26 INFO Signed narinfos id=46 count=1 worker2 # [ 148.292109] nix-grpc-daemon[1726]: time=2026-09-22T11:27:26.906Z level=INFO msg="Uploading 1 narinfos" lb # [ 148.302139] niks3-server[1098]: 2026/09/22 11:27:26 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 148.313128] niks3-server[1098]: 2026/09/22 11:27:26 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo lb # [ 148.323049] niks3-server[1098]: 2026/09/22 11:27:26 INFO Completed upload id=46 worker2 # [ 148.324729] nix-grpc-daemon[1726]: time=2026-09-22T11:27:26.939Z level=INFO msg="Upload complete. (254ms)" worker2 # [ 148.335362] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49346 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker2 # [ 148.355662] nix-daemon[836]: reaped child process 1699, status = succeeded client # [ 148.514040] systemd[1]: mid.service: Deactivated successfully. client # [ 148.518992] systemd[1]: mid.service: Consumed 204ms CPU time over 9.989s wall clock time, 43.4M memory peak, 9.1K incoming IP traffic, 11.3K outgoing IP traffic. lb # [ 148.560820] envoy[908]: conn peer=[2001:db8:1::4]:57214 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2666 tx=376 ms=17300 lb # [ 148.566309] envoy[908]: conn peer=[2001:db8:1::1]:44508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4288 tx=1948 ms=9769 client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.69 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 149.189649] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: c3af029464d541cf8c6f4a19e27bedb5 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 149.221301] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49346 client # [ 149.373289] nix[1433]: don't know how to build these paths: client # [ 149.374667] nix[1433]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 149.308475] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:27Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 149.314268] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:27Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 149.369738] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 149.391511] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 149.414542] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5) worker1 # [ 149.470568] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5) worker1 # [ 149.648616] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 3/5) worker1 # [ 149.657870] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 175 ms (attempt 4/5) worker1 # [ 149.835630] nix-grpc-daemon[1487]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 149.841727] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 149.860787] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 client # [ 149.960076] nix[1433]: copying 1 paths to 'lb:50051'... worker1 # [ 149.877565] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 client # [ 149.962567] nix[1433]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker1 # [ 149.892138] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 149.924860] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:28Z level=info event=niks3_push_started pid=1583 worker1 # [ 150.071896] nix-grpc-daemon[1583]: time=2026-09-22T11:27:28.682Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 150.185150] nix-daemon[847]: accepted connection from pid 1595, user nix-grpc-daemon (trusted) worker1 # [ 150.207749] nix-daemon[847]: reaped child process 1598, status = succeeded worker1 # [ 150.315056] nix-daemon[847]: accepted connection from pid 1600, user nix-grpc-daemon (trusted) worker1 # [ 150.346666] nix-daemon[847]: reaped child process 1606, status = succeeded lb # [ 150.285190] niks3-server[1098]: 2026/09/22 11:27:28 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 150.362021] nix-grpc-daemon[1583]: time=2026-09-22T11:27:28.971Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 150.363532] nix-grpc-daemon[1583]: time=2026-09-22T11:27:28.971Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 150.329236] niks3-server[1098]: 2026/09/22 11:27:28 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker1 # [ 150.404604] nix-grpc-daemon[1583]: time=2026-09-22T11:27:29.015Z level=INFO msg="Uploading 1 narinfos" lb # [ 150.342923] niks3-server[1098]: 2026/09/22 11:27:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 150.355497] niks3-server[1098]: 2026/09/22 11:27:28 INFO Signed narinfos id=47 count=1 lb # [ 150.361654] niks3-server[1098]: 2026/09/22 11:27:28 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 150.374350] niks3-server[1098]: 2026/09/22 11:27:28 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete worker1 # [ 150.443429] nix-grpc-daemon[1583]: time=2026-09-22T11:27:29.054Z level=INFO msg="Upload complete. (367ms)" lb # [ 150.382594] niks3-server[1098]: 2026/09/22 11:27:28 INFO Completed upload id=47 lb # [ 150.385849] niks3-server[1098]: 2026/09/22 11:27:28 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker1 # [ 150.454250] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 150.465981] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 150.474820] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 150.477643] nix-daemon[847]: reaped child process 1581, status = succeeded client # [ 150.570774] nix[1433]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... worker1 # [ 150.490322] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) client: must succeed: systemctl kill -s INT intr client # [ 152.841089] systemd[1]: intr.service: Sent signal SIGINT to main process 1433 (nix) on client request. worker1 # [ 152.762705] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:31Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 152.850284] nix[1433]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 152.865352] nix[1433]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds) client # [ 152.895898] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 152.901524] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 152.903942] systemd[1]: intr.service: Consumed 172ms CPU time over 3.709s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 152.918633] nix-daemon[1611]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 152.927166] nix-daemon[847]: reaped child process 1609, status = succeeded worker1 # [ 152.928311] nix-daemon[847]: reaped child process 1611, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 4.98 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 154.201247] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: bbc75801cc824b828fd5538cd4a2c1df client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds) worker2 # [ 154.263675] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49346 client # [ 154.414950] nix[1454]: don't know how to build these paths: client # [ 154.416282] nix[1454]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 154.345981] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:32Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 154.351877] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:32Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 154.329897] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49346 worker2 # [ 154.344690] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5) worker2 # [ 154.410033] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5) worker2 # [ 154.496788] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 345 ms (attempt 3/5) worker2 # [ 154.844113] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 130 ms (attempt 4/5) worker2 # [ 154.976506] nix-grpc-daemon[1645]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 154.982945] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 155.000932] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49346 client # [ 155.160231] nix[1454]: copying 1 paths to 'lb:50051'... worker2 # [ 155.016558] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49346 client # [ 155.160532] nix[1454]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker2 # [ 155.031803] nix-daemon[836]: accepted connection from pid 1645, user nix-grpc-daemon (trusted) worker2 # [ 155.193461] nix-daemon[836]: accepted connection from pid 1778, user nix-grpc-daemon (trusted) worker2 # [ 155.215737] nix-daemon[836]: reaped child process 1781, status = succeeded worker2 # [ 155.317908] nix-daemon[836]: accepted connection from pid 1783, user nix-grpc-daemon (trusted) worker2 # [ 155.347543] nix-daemon[836]: reaped child process 1789, status = succeeded lb # [ 155.345084] niks3-server[1098]: 2026/09/22 11:27:33 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 155.356321] nix-grpc-daemon[1726]: time=2026-09-22T11:27:33.970Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 155.357814] nix-grpc-daemon[1726]: time=2026-09-22T11:27:33.970Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 155.378700] niks3-server[1098]: 2026/09/22 11:27:33 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst worker2 # [ 155.400989] nix-grpc-daemon[1726]: time=2026-09-22T11:27:34.015Z level=INFO msg="Uploading 1 narinfos" lb # [ 155.399285] niks3-server[1098]: 2026/09/22 11:27:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 155.405822] niks3-server[1098]: 2026/09/22 11:27:34 INFO Signed narinfos id=48 count=1 lb # [ 155.413604] niks3-server[1098]: 2026/09/22 11:27:34 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 155.427222] niks3-server[1098]: 2026/09/22 11:27:34 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 155.437095] niks3-server[1098]: 2026/09/22 11:27:34 INFO Completed upload id=48 worker2 # [ 155.438620] nix-grpc-daemon[1726]: time=2026-09-22T11:27:34.053Z level=INFO msg="Upload complete. (389ms)" worker2 # [ 155.447759] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49346 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 155.444806] niks3-server[1098]: 2026/09/22 11:27:34 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker2 # [ 155.459604] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49346 worker2 # [ 155.468218] nix-daemon[836]: accepted connection from pid 1645, user nix-grpc-daemon (trusted) worker2 # [ 155.470799] nix-daemon[836]: reaped child process 1776, status = succeeded client # [ 155.625128] nix[1454]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker2 # [ 155.483318] nix-daemon[836]: accepted connection from pid 1645, user nix-grpc-daemon (trusted) worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2 worker2 # Checking switch inhibitors... done worker2 # [ 158.205547] nixos[1826]: switching to system configuration /nix/store/mms4r3ic4vc4scxwax7lpjppqd3q2hgx-nixos-system-worker2-test worker2 # [ 158.207378] systemd[1]: Stopped target Remote File Systems. worker2 # activating the configuration... worker2 # [ 158.213339] systemd[1]: Stopped target Local File Systems. worker2 # setting up /etc... lb # [ 158.561214] envoy[908]: conn peer=[2001:db8:1::1]:49366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=827 ms=3509 worker2 # [ 158.761548] systemd[1]: Reload requested from client PID 1826 ('.switch-to-conf') (unit backdoor.service)... worker2 # [ 158.763104] systemd[1]: Reloading... worker2 # [ 163.653673] systemd[1]: Reloading finished in 4884 ms. worker2 # restarting sysinit-reactivation.target worker2 # [ 163.768728] systemd[1]: Starting Re-setup tmpfiles on a system that is already running.... worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service[ 163.892457] systemd[1]: Finished Re-setup tmpfiles on a system that is already running.. worker2 # [ 163.893606] systemd[1]: Reached target Reactivate sysinit units. worker2 # worker2 # [ 163.907579] systemd[1]: Reloading D-Bus System Message Bus... worker2 # [ 163.910922] dbus-broker-launch[491]: Caught SIGHUP, trigger reload. worker2 # [ 163.911890] systemd[1]: Reloading Nix worker-protocol over gRPC... worker2 # [ 163.920736] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 163.922855] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync' worker2 # [ 163.924257] dbus-broker-launch[491]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 163.951913] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker2 # [ 163.959029] systemd[1]: Reloaded D-Bus System Message Bus. worker2 # [ 163.971306] systemd[1]: Reached target Local File Systems. worker2 # [ 163.978376] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 163.979842] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 163.992228] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 163.993490] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 163.995535] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 164.003391] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 164.009873] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 164.011485] systemd[1]: Reached target Remote File Systems. worker2 # [ 164.017046] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 164.018274] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 164.307905] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 164.309572] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 164.316307] systemd[1]: Starting linger-users.service... worker2 # [ 164.359070] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 164.360191] systemd[1]: Finished linger-users.service. worker2 # [ 164.508306] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:43Z level=info event=draining worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service worker2 # [ 164.612633] nixos[1826]: finished switching to system configuration /nix/store/mms4r3ic4vc4scxwax7lpjppqd3q2hgx-nixos-system-worker2-test worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.86 seconds) worker2: must succeed: pgrep -f 'read -t [9]0 x' worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 164.955999] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 164.963520] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 164.991289] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:43Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 164.998377] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:43Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 165.039102] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 165.054425] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker1 # [ 165.070492] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 165.085662] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 165.239170] nix-daemon[847]: accepted connection from pid 1670, user nix-grpc-daemon (trusted) worker1 # [ 165.259550] nix-daemon[847]: reaped child process 1673, status = succeeded worker1 # [ 165.357574] nix-daemon[847]: accepted connection from pid 1675, user nix-grpc-daemon (trusted) worker1 # [ 165.373304] nix-daemon[847]: reaped child process 1678, status = succeeded lb # [ 165.317357] niks3-server[1098]: 2026/09/22 11:27:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 165.390937] nix-grpc-daemon[1583]: time=2026-09-22T11:27:44.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 165.392607] nix-grpc-daemon[1583]: time=2026-09-22T11:27:44.001Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 165.356513] niks3-server[1098]: 2026/09/22 11:27:43 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst worker1 # [ 165.444277] nix-grpc-daemon[1583]: time=2026-09-22T11:27:44.055Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.377510] niks3-server[1098]: 2026/09/22 11:27:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 165.387269] niks3-server[1098]: 2026/09/22 11:27:43 INFO Signed narinfos id=49 count=1 lb # [ 165.392539] niks3-server[1098]: 2026/09/22 11:27:43 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 165.404094] niks3-server[1098]: 2026/09/22 11:27:44 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 165.410291] niks3-server[1098]: 2026/09/22 11:27:44 INFO Completed upload id=49 worker1 # [ 165.473059] nix-grpc-daemon[1583]: time=2026-09-22T11:27:44.084Z level=INFO msg="Upload complete. (370ms)" lb # [ 165.413742] niks3-server[1098]: 2026/09/22 11:27:44 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker1 # [ 165.485373] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=528 worker1 # [ 165.500339] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 165.507701] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 165.513486] nix-daemon[847]: reaped child process 1668, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker1 # [ 165.528441] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker1 # [ 166.773054] nix-daemon[847]: reaped child process 1682, status = succeeded worker1 # [ 166.889186] nix-daemon[847]: accepted connection from pid 1698, user nix-grpc-daemon (trusted) worker1 # [ 166.911844] nix-daemon[847]: reaped child process 1701, status = succeeded lb # [ 166.851577] niks3-server[1098]: 2026/09/22 11:27:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 166.927603] nix-grpc-daemon[1583]: time=2026-09-22T11:27:45.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 166.930398] nix-grpc-daemon[1583]: time=2026-09-22T11:27:45.535Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 166.890765] niks3-server[1098]: 2026/09/22 11:27:45 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst worker1 # [ 166.974725] nix-grpc-daemon[1583]: time=2026-09-22T11:27:45.584Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.910872] niks3-server[1098]: 2026/09/22 11:27:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 166.923458] niks3-server[1098]: 2026/09/22 11:27:45 INFO Signed narinfos id=50 count=1 lb # [ 166.929595] niks3-server[1098]: 2026/09/22 11:27:45 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 166.938555] niks3-server[1098]: 2026/09/22 11:27:45 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 166.948295] niks3-server[1098]: 2026/09/22 11:27:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 166.955622] niks3-server[1098]: 2026/09/22 11:27:45 INFO Completed upload id=50 worker1 # [ 167.018270] nix-grpc-daemon[1583]: time=2026-09-22T11:27:45.629Z level=INFO msg="Upload complete. (278ms)" lb # [ 166.959178] niks3-server[1098]: 2026/09/22 11:27:45 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker1 # [ 167.037455] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker1 # [ 167.052327] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:45Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 167.058202] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:45Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 167.062467] nix-daemon[847]: reaped child process 1680, status = succeeded worker1 # [ 167.083302] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 167.098918] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 167.114428] nix-grpc-daemon[1583]: time=2026-09-22T11:27:45.725Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 167.125009] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker1 # [ 167.140501] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 167.299035] nix-daemon[847]: accepted connection from pid 1707, user nix-grpc-daemon (trusted) worker1 # [ 167.318027] nix-daemon[847]: reaped child process 1710, status = succeeded worker1 # [ 167.423873] nix-daemon[847]: accepted connection from pid 1712, user nix-grpc-daemon (trusted) worker1 # [ 167.445065] nix-daemon[847]: reaped child process 1715, status = succeeded lb # [ 167.380494] niks3-server[1098]: 2026/09/22 11:27:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 167.452949] nix-grpc-daemon[1583]: time=2026-09-22T11:27:46.063Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 167.454458] nix-grpc-daemon[1583]: time=2026-09-22T11:27:46.063Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 167.410238] niks3-server[1098]: 2026/09/22 11:27:46 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker1 # [ 167.488483] nix-grpc-daemon[1583]: time=2026-09-22T11:27:46.099Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.425584] niks3-server[1098]: 2026/09/22 11:27:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 167.436005] niks3-server[1098]: 2026/09/22 11:27:46 INFO Signed narinfos id=51 count=1 lb # [ 167.443950] niks3-server[1098]: 2026/09/22 11:27:46 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 167.458215] niks3-server[1098]: 2026/09/22 11:27:46 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete worker1 # [ 167.532682] nix-grpc-daemon[1583]: time=2026-09-22T11:27:46.143Z level=INFO msg="Upload complete. (375ms)" lb # [ 167.475263] niks3-server[1098]: 2026/09/22 11:27:46 INFO Completed upload id=51 worker1 # [ 167.540318] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 167.479691] niks3-server[1098]: 2026/09/22 11:27:46 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker1 # [ 167.556235] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 167.563845] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 167.569608] nix-daemon[847]: reaped child process 1705, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker1 # [ 167.584192] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker1 # [ 168.428690] nix-daemon[847]: reaped child process 1719, status = succeeded worker1 # [ 168.541213] nix-daemon[847]: accepted connection from pid 1735, user nix-grpc-daemon (trusted) worker1 # [ 168.563002] nix-daemon[847]: reaped child process 1738, status = succeeded lb # [ 168.499104] niks3-server[1098]: 2026/09/22 11:27:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 168.573554] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.183Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 168.575063] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.183Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 168.537839] niks3-server[1098]: 2026/09/22 11:27:47 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 168.553772] niks3-server[1098]: 2026/09/22 11:27:47 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst worker1 # [ 168.631201] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.241Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.568884] niks3-server[1098]: 2026/09/22 11:27:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 168.576350] niks3-server[1098]: 2026/09/22 11:27:47 INFO Signed narinfos id=52 count=1 lb # [ 168.587914] niks3-server[1098]: 2026/09/22 11:27:47 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 168.595382] niks3-server[1098]: 2026/09/22 11:27:47 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 168.604318] niks3-server[1098]: 2026/09/22 11:27:47 INFO Completed upload id=52 worker1 # [ 168.666784] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.277Z level=INFO msg="Upload complete. (272ms)" lb # [ 168.607345] niks3-server[1098]: 2026/09/22 11:27:47 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 168.689106] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker1 # [ 168.704379] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker1 # [ 168.709353] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 168.713801] nix-daemon[847]: reaped child process 1717, status = succeeded worker1 # [ 168.733804] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 168.750696] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 168.767435] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.378Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker1 # [ 168.778519] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 168.795028] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 168.951439] nix-daemon[847]: accepted connection from pid 1744, user nix-grpc-daemon (trusted) worker1 # [ 168.971445] nix-daemon[847]: reaped child process 1747, status = succeeded worker1 # [ 169.077049] nix-daemon[847]: accepted connection from pid 1749, user nix-grpc-daemon (trusted) worker1 # [ 169.094138] nix-daemon[847]: reaped child process 1752, status = succeeded lb # [ 169.039248] niks3-server[1098]: 2026/09/22 11:27:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 169.111006] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.721Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 169.112638] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.721Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 169.072172] niks3-server[1098]: 2026/09/22 11:27:47 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 169.087670] niks3-server[1098]: 2026/09/22 11:27:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign worker1 # [ 169.158282] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.769Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.100905] niks3-server[1098]: 2026/09/22 11:27:47 INFO Signed narinfos id=53 count=1 lb # [ 169.107996] niks3-server[1098]: 2026/09/22 11:27:47 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 169.123266] niks3-server[1098]: 2026/09/22 11:27:47 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 169.133540] niks3-server[1098]: 2026/09/22 11:27:47 INFO Completed upload id=53 worker1 # [ 169.198098] nix-grpc-daemon[1583]: time=2026-09-22T11:27:47.807Z level=INFO msg="Upload complete. (383ms)" lb # [ 169.141933] niks3-server[1098]: 2026/09/22 11:27:47 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker1 # [ 169.209344] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 169.224641] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 169.232474] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 169.237583] nix-daemon[847]: reaped child process 1742, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker1 # [ 169.252703] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker1 # [ 170.102769] nix-daemon[847]: reaped child process 1756, status = succeeded worker1 # [ 170.207729] nix-daemon[847]: accepted connection from pid 1772, user nix-grpc-daemon (trusted) worker1 # [ 170.224763] nix-daemon[847]: reaped child process 1775, status = succeeded lb # [ 170.169239] niks3-server[1098]: 2026/09/22 11:27:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 170.244486] nix-grpc-daemon[1583]: time=2026-09-22T11:27:48.853Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 170.246403] nix-grpc-daemon[1583]: time=2026-09-22T11:27:48.853Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 170.203589] niks3-server[1098]: 2026/09/22 11:27:48 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 170.218380] niks3-server[1098]: 2026/09/22 11:27:48 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 170.236291] niks3-server[1098]: 2026/09/22 11:27:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 170.304556] nix-grpc-daemon[1583]: time=2026-09-22T11:27:48.915Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.244848] niks3-server[1098]: 2026/09/22 11:27:48 INFO Signed narinfos id=54 count=1 lb # [ 170.250185] niks3-server[1098]: 2026/09/22 11:27:48 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 170.258472] niks3-server[1098]: 2026/09/22 11:27:48 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 170.264908] niks3-server[1098]: 2026/09/22 11:27:48 INFO Completed upload id=54 worker1 # [ 170.327505] nix-grpc-daemon[1583]: time=2026-09-22T11:27:48.938Z level=INFO msg="Upload complete. (264ms)" lb # [ 170.268241] niks3-server[1098]: 2026/09/22 11:27:48 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker1 # [ 170.341136] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker1 # [ 170.357691] nix-daemon[847]: reaped child process 1754, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.65 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds) worker2: must succeed: pkill -f 'read -t [9]0 x' worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker2 # [ 170.770142] nix-daemon[836]: reaped child process 1794, status = succeeded worker2 # [ 170.880125] nix-daemon[836]: accepted connection from pid 1966, user nix-grpc-daemon (trusted) worker2 # [ 170.901463] nix-daemon[836]: reaped child process 1969, status = succeeded lb # [ 170.900489] niks3-server[1098]: 2026/09/22 11:27:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 170.912576] nix-grpc-daemon[1726]: time=2026-09-22T11:27:49.526Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 170.914171] nix-grpc-daemon[1726]: time=2026-09-22T11:27:49.526Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 170.939947] niks3-server[1098]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 170.954950] niks3-server[1098]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker2 # [ 170.966266] nix-grpc-daemon[1726]: time=2026-09-22T11:27:49.579Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.966243] niks3-server[1098]: 2026/09/22 11:27:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 170.974622] niks3-server[1098]: 2026/09/22 11:27:49 INFO Signed narinfos id=55 count=1 lb # [ 170.978868] niks3-server[1098]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 170.986042] niks3-server[1098]: 2026/09/22 11:27:49 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 170.990536] niks3-server[1098]: 2026/09/22 11:27:49 INFO Completed upload id=55 worker2 # [ 170.992133] nix-grpc-daemon[1726]: time=2026-09-22T11:27:49.606Z level=INFO msg="Upload complete. (283ms)" lb # [ 170.993164] niks3-server[1098]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker2 # [ 171.005109] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49346 duration_s=15 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 worker2 # [ 171.022715] nix-daemon[836]: reaped child process 1792, status = succeeded client # [ 171.179751] systemd[1]: sw.service: Deactivated successfully. client # [ 171.184122] systemd[1]: sw.service: Consumed 198ms CPU time over 16.982s wall clock time, 39.4M memory peak, 7.5K incoming IP traffic, 8.7K outgoing IP traffic. worker2 # [ 171.510751] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:50Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 171.782248] systemd[1]: Starting scheduler-token.service... worker2 # [ 171.853492] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 171.858854] systemd[1]: Finished scheduler-token.service. worker2 # [ 171.860658] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 75ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker1 # [ 172.584522] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:46092 worker2 # [ 172.525531] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:51Z level=info event=scheduler_restarting worker1 # [ 172.584900] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:51Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 172.550376] nix-grpc-daemon[1645]: ts=2026-09-22T11:27:51Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 172.575014] nix-daemon[836]: reaped child process 1697, status = succeeded worker2 # [ 172.680769] nix-grpc-daemon[1645]: E0000 00:00:1790076471.295291 1690 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 172.687795] nix-grpc-daemon[1645]: E0000 00:00:1790076471.295491 1690 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 174.557596] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 174.563739] systemd[1]: nix-grpc-daemon.service: Consumed 897ms CPU time over 32.283s wall clock time, 35.4M memory peak, 21.6K incoming IP traffic, 35.1K outgoing IP traffic. worker2 # [ 174.592625] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 174.735811] nix-grpc-daemon[1986]: ts=2026-09-22T11:27:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 174.738303] nix-grpc-daemon[1986]: ts=2026-09-22T11:27:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 174.742157] nix-grpc-daemon[1986]: ts=2026-09-22T11:27:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 174.747422] nix-grpc-daemon[1986]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 174.764244] nix-grpc-daemon[1986]: E0000 00:00:1790076473.361514 1986 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.10 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 174.795959] nix-grpc-daemon[1986]: E0000 00:00:1790076473.365107 1986 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 174.798878] nix-grpc-daemon[1986]: ts=2026-09-22T11:27:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 174.800933] nix-grpc-daemon[1986]: E0000 00:00:1790076473.373309 2028 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 174.803218] nix-grpc-daemon[1986]: E0000 00:00:1790076473.394136 2028 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 174.805603] nix-grpc-daemon[1986]: E0000 00:00:1790076473.394279 2028 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 174.810647] nix-grpc-daemon[1986]: E0000 00:00:1790076473.409617 2028 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 174.815237] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 174.816286] nix-grpc-daemon[1986]: ts=2026-09-22T11:27:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 174.880917] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:46092 worker1 # [ 174.881290] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 22.08 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 176.299761] nix-grpc-daemon[1986]: ts=2026-09-22T11:27:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 176.313878] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 176.334694] nix-grpc-daemon[1986]: ts=2026-09-22T11:27:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 176.350048] nix-grpc-daemon[1986]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5) worker2 # [ 176.447403] nix-grpc-daemon[1986]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 2/5) worker2 # [ 176.590273] nix-grpc-daemon[1986]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 3/5) worker2 # [ 176.797306] nix-grpc-daemon[1986]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 342 ms (attempt 4/5) worker2 # [ 177.141746] nix-grpc-daemon[1986]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 177.149082] nix-grpc-daemon[1986]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 177.263858] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:55Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker1 # [ 177.269489] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:55Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 177.275501] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:55Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker1 # [ 177.280915] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:55Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 177.306997] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 177.323133] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 177.337959] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 177.353121] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 177.515726] nix-daemon[847]: accepted connection from pid 1784, user nix-grpc-daemon (trusted) worker1 # [ 177.537339] nix-daemon[847]: reaped child process 1787, status = succeeded worker1 # [ 177.632671] nix-daemon[847]: accepted connection from pid 1789, user nix-grpc-daemon (trusted) worker1 # [ 177.652296] nix-daemon[847]: reaped child process 1792, status = succeeded lb # [ 177.589421] niks3-server[1098]: 2026/09/22 11:27:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 177.661979] nix-grpc-daemon[1583]: time=2026-09-22T11:27:56.272Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 177.663420] nix-grpc-daemon[1583]: time=2026-09-22T11:27:56.272Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 177.617052] niks3-server[1098]: 2026/09/22 11:27:56 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 177.701774] nix-grpc-daemon[1583]: time=2026-09-22T11:27:56.312Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.632983] niks3-server[1098]: 2026/09/22 11:27:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 177.642145] niks3-server[1098]: 2026/09/22 11:27:56 INFO Signed narinfos id=56 count=1 lb # [ 177.653781] niks3-server[1098]: 2026/09/22 11:27:56 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 177.732322] nix-grpc-daemon[1583]: time=2026-09-22T11:27:56.343Z level=INFO msg="Upload complete. (362ms)" lb # [ 177.664367] niks3-server[1098]: 2026/09/22 11:27:56 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 177.673294] niks3-server[1098]: 2026/09/22 11:27:56 INFO Completed upload id=56 worker1 # [ 177.740189] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 177.680222] niks3-server[1098]: 2026/09/22 11:27:56 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 177.756762] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 177.763974] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 177.769520] nix-daemon[847]: reaped child process 1782, status = succeeded worker1 # [ 177.780129] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) lb # [ 178.559977] envoy[908]: conn peer=[2001:db8:1::1]:37022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9634 tx=4038 ms=5396 lb # [ 178.560354] envoy[908]: conn peer=[2001:db8:1::1]:55982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3025 tx=1466 ms=16753 lb # [ 178.560476] envoy[908]: conn peer=[2001:db8:1::4]:41580 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7565 tx=668 ms=29973 worker1 # [ 178.889749] nix-daemon[847]: reaped child process 1796, status = succeeded worker1 # [ 178.962591] nix-daemon[847]: accepted connection from pid 1812, user nix-grpc-daemon (trusted) worker1 # [ 178.982311] nix-daemon[847]: reaped child process 1815, status = succeeded lb # [ 178.919062] niks3-server[1098]: 2026/09/22 11:27:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 178.992620] nix-grpc-daemon[1583]: time=2026-09-22T11:27:57.602Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 178.994668] nix-grpc-daemon[1583]: time=2026-09-22T11:27:57.602Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" worker1 # [ 179.035457] nix-grpc-daemon[1583]: time=2026-09-22T11:27:57.646Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.965635] niks3-server[1098]: 2026/09/22 11:27:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 178.977123] niks3-server[1098]: 2026/09/22 11:27:57 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 178.990238] niks3-server[1098]: 2026/09/22 11:27:57 INFO Signed narinfos id=57 count=1 lb # [ 178.995386] niks3-server[1098]: 2026/09/22 11:27:57 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls worker1 # [ 179.066842] nix-grpc-daemon[1583]: time=2026-09-22T11:27:57.677Z level=INFO msg="Upload complete. (275ms)" lb # [ 179.006523] niks3-server[1098]: 2026/09/22 11:27:57 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 179.018274] niks3-server[1098]: 2026/09/22 11:27:57 INFO Completed upload id=57 lb # [ 179.023315] niks3-server[1098]: 2026/09/22 11:27:57 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 179.086457] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=1 lb # [ 179.028758] niks3-server[1098]: 2026/09/22 11:27:57 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker1 # [ 179.101108] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:57Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 179.111007] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 179.115724] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:57Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 179.132374] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 179.135718] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 179.142309] nix-daemon[847]: reaped child process 1794, status = succeeded worker1 # [ 179.144185] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 179.267035] nix-daemon[847]: accepted connection from pid 1821, user nix-grpc-daemon (trusted) worker1 # [ 179.283440] nix-daemon[847]: reaped child process 1824, status = succeeded worker1 # [ 179.383711] nix-daemon[847]: accepted connection from pid 1826, user nix-grpc-daemon (trusted) worker1 # [ 179.398568] nix-daemon[847]: reaped child process 1829, status = succeeded lb # [ 179.342844] niks3-server[1098]: 2026/09/22 11:27:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 179.418767] nix-grpc-daemon[1583]: time=2026-09-22T11:27:58.029Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 179.420329] nix-grpc-daemon[1583]: time=2026-09-22T11:27:58.029Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" worker1 # [ 179.454535] nix-grpc-daemon[1583]: time=2026-09-22T11:27:58.065Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.381233] niks3-server[1098]: 2026/09/22 11:27:57 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst lb # [ 179.397041] niks3-server[1098]: 2026/09/22 11:27:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 179.406400] niks3-server[1098]: 2026/09/22 11:27:57 INFO Signed narinfos id=58 count=1 lb # [ 179.410762] niks3-server[1098]: 2026/09/22 11:27:58 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls lb # [ 179.421304] niks3-server[1098]: 2026/09/22 11:27:58 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 179.426202] niks3-server[1098]: 2026/09/22 11:27:58 INFO Completed upload id=58 worker1 # [ 179.487414] nix-grpc-daemon[1583]: time=2026-09-22T11:27:58.098Z level=INFO msg="Upload complete. (346ms)" lb # [ 179.428631] niks3-server[1098]: 2026/09/22 11:27:58 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 179.497922] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 179.512108] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 179.521154] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 179.524800] nix-daemon[847]: reaped child process 1819, status = succeeded worker1 # [ 179.536165] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 180.773788] nix-daemon[847]: reaped child process 1833, status = succeeded worker1 # [ 180.868437] nix-daemon[847]: accepted connection from pid 1849, user nix-grpc-daemon (trusted) worker1 # [ 180.885613] nix-daemon[847]: reaped child process 1852, status = succeeded lb # [ 180.830937] niks3-server[1098]: 2026/09/22 11:27:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 180.904394] nix-grpc-daemon[1583]: time=2026-09-22T11:27:59.513Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 180.905885] nix-grpc-daemon[1583]: time=2026-09-22T11:27:59.513Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 180.873041] niks3-server[1098]: 2026/09/22 11:27:59 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 180.948815] nix-grpc-daemon[1583]: time=2026-09-22T11:27:59.559Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.886709] niks3-server[1098]: 2026/09/22 11:27:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 180.897648] niks3-server[1098]: 2026/09/22 11:27:59 INFO Signed narinfos id=59 count=1 lb # [ 180.903059] niks3-server[1098]: 2026/09/22 11:27:59 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 180.911363] niks3-server[1098]: 2026/09/22 11:27:59 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 180.920604] niks3-server[1098]: 2026/09/22 11:27:59 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 180.926040] niks3-server[1098]: 2026/09/22 11:27:59 INFO Completed upload id=59 worker1 # [ 180.988797] nix-grpc-daemon[1583]: time=2026-09-22T11:27:59.599Z level=INFO msg="Upload complete. (241ms)" lb # [ 180.929703] niks3-server[1098]: 2026/09/22 11:27:59 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 181.006485] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=1 worker1 # [ 181.024764] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:59Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 181.031473] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 181.036514] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:59Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 181.040944] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 181.055633] nix-daemon[847]: reaped child process 1831, status = succeeded worker1 # [ 181.057674] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 181.059859] nix-grpc-daemon[1487]: ts=2026-09-22T11:27:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 181.185597] nix-daemon[847]: accepted connection from pid 1858, user nix-grpc-daemon (trusted) worker1 # [ 181.202179] nix-daemon[847]: reaped child process 1861, status = succeeded worker1 # [ 181.307630] nix-daemon[847]: accepted connection from pid 1863, user nix-grpc-daemon (trusted) worker1 # [ 181.322690] nix-daemon[847]: reaped child process 1866, status = succeeded lb # [ 181.267401] niks3-server[1098]: 2026/09/22 11:27:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 181.339732] nix-grpc-daemon[1583]: time=2026-09-22T11:27:59.950Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 181.341408] nix-grpc-daemon[1583]: time=2026-09-22T11:27:59.950Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 181.293924] niks3-server[1098]: 2026/09/22 11:27:59 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 181.309945] niks3-server[1098]: 2026/09/22 11:27:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker1 # [ 181.378850] nix-grpc-daemon[1583]: time=2026-09-22T11:27:59.989Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.322679] niks3-server[1098]: 2026/09/22 11:27:59 INFO Signed narinfos id=60 count=1 lb # [ 181.329926] niks3-server[1098]: 2026/09/22 11:27:59 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls worker1 # [ 181.403643] nix-grpc-daemon[1583]: time=2026-09-22T11:28:00.014Z level=INFO msg="Upload complete. (338ms)" lb # [ 181.341885] niks3-server[1098]: 2026/09/22 11:27:59 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 181.350226] niks3-server[1098]: 2026/09/22 11:27:59 INFO Completed upload id=60 worker1 # [ 181.411202] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 181.352721] niks3-server[1098]: 2026/09/22 11:27:59 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 181.424000] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 181.432308] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 181.435020] nix-daemon[847]: reaped child process 1856, status = succeeded worker1 # [ 181.448292] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 182.703689] nix-daemon[847]: reaped child process 1870, status = succeeded worker1 # [ 182.828841] nix-daemon[847]: accepted connection from pid 1886, user nix-grpc-daemon (trusted) worker1 # [ 182.846489] nix-daemon[847]: reaped child process 1889, status = succeeded lb # [ 182.789253] niks3-server[1098]: 2026/09/22 11:28:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 182.862607] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.472Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 182.864338] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.472Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 182.823553] niks3-server[1098]: 2026/09/22 11:28:01 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 182.837131] niks3-server[1098]: 2026/09/22 11:28:01 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 182.848746] niks3-server[1098]: 2026/09/22 11:28:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign worker1 # [ 182.916980] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.527Z level=INFO msg="Uploading 1 narinfos" lb # [ 182.858394] niks3-server[1098]: 2026/09/22 11:28:01 INFO Signed narinfos id=61 count=1 lb # [ 182.862838] niks3-server[1098]: 2026/09/22 11:28:01 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 182.871755] niks3-server[1098]: 2026/09/22 11:28:01 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 182.878955] niks3-server[1098]: 2026/09/22 11:28:01 INFO Completed upload id=61 lb # [ 182.882059] niks3-server[1098]: 2026/09/22 11:28:01 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker1 # [ 182.933005] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.544Z level=INFO msg="Upload complete. (274ms)" worker1 # [ 182.951778] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=1 worker1 # [ 182.969303] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:01Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 182.974333] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:01Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 182.978462] nix-daemon[847]: reaped child process 1868, status = succeeded worker1 # [ 182.991417] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 183.006189] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 183.019495] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.630Z level=INFO msg="All 1 paths already cached" worker1 # [ 183.028111] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 183.041463] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 183.193412] nix-daemon[847]: accepted connection from pid 1895, user nix-grpc-daemon (trusted) worker1 # [ 183.215911] nix-daemon[847]: reaped child process 1898, status = succeeded worker1 # [ 183.311303] nix-daemon[847]: accepted connection from pid 1900, user nix-grpc-daemon (trusted) worker1 # [ 183.325978] nix-daemon[847]: reaped child process 1903, status = succeeded lb # [ 183.270898] niks3-server[1098]: 2026/09/22 11:28:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.342007] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.952Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 183.344525] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.952Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 183.298618] niks3-server[1098]: 2026/09/22 11:28:01 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst lb # [ 183.311806] niks3-server[1098]: 2026/09/22 11:28:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 183.381395] nix-grpc-daemon[1583]: time=2026-09-22T11:28:01.992Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.324363] niks3-server[1098]: 2026/09/22 11:28:01 INFO Signed narinfos id=62 count=1 lb # [ 183.327048] niks3-server[1098]: 2026/09/22 11:28:01 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 183.339743] niks3-server[1098]: 2026/09/22 11:28:01 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 183.347849] niks3-server[1098]: 2026/09/22 11:28:01 INFO Completed upload id=62 worker1 # [ 183.410653] nix-grpc-daemon[1583]: time=2026-09-22T11:28:02.021Z level=INFO msg="Upload complete. (354ms)" lb # [ 183.351189] niks3-server[1098]: 2026/09/22 11:28:01 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 183.420344] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 183.433908] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 183.442205] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 183.445640] nix-daemon[847]: reaped child process 1893, status = succeeded worker1 # [ 183.455719] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 184.713652] nix-daemon[847]: reaped child process 1907, status = succeeded worker1 # [ 184.847861] nix-daemon[847]: accepted connection from pid 1923, user nix-grpc-daemon (trusted) worker1 # [ 184.867494] nix-daemon[847]: reaped child process 1926, status = succeeded lb # [ 184.811027] niks3-server[1098]: 2026/09/22 11:28:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 184.888458] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.495Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 184.890276] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.495Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 184.853595] niks3-server[1098]: 2026/09/22 11:28:03 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 184.864788] niks3-server[1098]: 2026/09/22 11:28:03 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker1 # [ 184.939554] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.550Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.878487] niks3-server[1098]: 2026/09/22 11:28:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 184.890177] niks3-server[1098]: 2026/09/22 11:28:03 INFO Signed narinfos id=63 count=1 lb # [ 184.895632] niks3-server[1098]: 2026/09/22 11:28:03 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 184.904726] niks3-server[1098]: 2026/09/22 11:28:03 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 184.912890] niks3-server[1098]: 2026/09/22 11:28:03 INFO Completed upload id=63 worker1 # [ 184.975901] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.586Z level=INFO msg="Upload complete. (307ms)" lb # [ 184.916557] niks3-server[1098]: 2026/09/22 11:28:03 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 184.997211] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=1 worker1 # [ 185.012856] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:03Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 185.022372] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 185.027904] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:03Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 185.033236] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 185.040263] nix-daemon[847]: reaped child process 1905, status = succeeded worker1 # [ 185.042421] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.643Z level=INFO msg="All 1 paths already cached" worker1 # [ 185.051628] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 185.060183] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 185.185117] nix-daemon[847]: accepted connection from pid 1932, user nix-grpc-daemon (trusted) worker1 # [ 185.208514] nix-daemon[847]: reaped child process 1935, status = succeeded worker1 # [ 185.302216] nix-daemon[847]: accepted connection from pid 1937, user nix-grpc-daemon (trusted) worker1 # [ 185.318044] nix-daemon[847]: reaped child process 1940, status = succeeded lb # [ 185.261136] niks3-server[1098]: 2026/09/22 11:28:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.332279] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.943Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 185.333727] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.943Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 185.289526] niks3-server[1098]: 2026/09/22 11:28:03 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst lb # [ 185.303317] niks3-server[1098]: 2026/09/22 11:28:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign worker1 # [ 185.371145] nix-grpc-daemon[1583]: time=2026-09-22T11:28:03.982Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.314720] niks3-server[1098]: 2026/09/22 11:28:03 INFO Signed narinfos id=64 count=1 lb # [ 185.320982] niks3-server[1098]: 2026/09/22 11:28:03 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls lb # [ 185.329561] niks3-server[1098]: 2026/09/22 11:28:03 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete worker1 # [ 185.396271] nix-grpc-daemon[1583]: time=2026-09-22T11:28:04.007Z level=INFO msg="Upload complete. (331ms)" lb # [ 185.335847] niks3-server[1098]: 2026/09/22 11:28:03 INFO Completed upload id=64 lb # [ 185.339342] niks3-server[1098]: 2026/09/22 11:28:03 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 185.404143] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 185.415984] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 185.421679] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 185.426252] nix-daemon[847]: reaped child process 1930, status = succeeded worker1 # [ 185.439124] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 186.749881] nix-daemon[847]: reaped child process 1944, status = succeeded worker1 # [ 186.817615] nix-daemon[847]: accepted connection from pid 1960, user nix-grpc-daemon (trusted) worker1 # [ 186.838102] nix-daemon[847]: reaped child process 1963, status = succeeded lb # [ 186.782057] niks3-server[1098]: 2026/09/22 11:28:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 186.852407] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.462Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 186.854127] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.462Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 186.818065] niks3-server[1098]: 2026/09/22 11:28:05 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 186.833394] niks3-server[1098]: 2026/09/22 11:28:05 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst worker1 # [ 186.907686] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.518Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.848306] niks3-server[1098]: 2026/09/22 11:28:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 186.857573] niks3-server[1098]: 2026/09/22 11:28:05 INFO Signed narinfos id=65 count=1 lb # [ 186.864566] niks3-server[1098]: 2026/09/22 11:28:05 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls worker1 # [ 186.939429] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.550Z level=INFO msg="Upload complete. (289ms)" lb # [ 186.875573] niks3-server[1098]: 2026/09/22 11:28:05 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 186.882098] niks3-server[1098]: 2026/09/22 11:28:05 INFO Completed upload id=65 lb # [ 186.885967] niks3-server[1098]: 2026/09/22 11:28:05 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 186.958482] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=1 worker1 # [ 186.980303] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:05Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 186.987951] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 186.995090] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:05Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 187.001652] nix-daemon[847]: reaped child process 1942, status = succeeded worker1 # [ 187.005179] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 187.018113] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.615Z level=INFO msg="All 1 paths already cached" worker1 # [ 187.025663] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 187.031851] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 187.161240] nix-daemon[847]: accepted connection from pid 1969, user nix-grpc-daemon (trusted) worker1 # [ 187.177900] nix-daemon[847]: reaped child process 1972, status = succeeded worker1 # [ 187.279250] nix-daemon[847]: accepted connection from pid 1974, user nix-grpc-daemon (trusted) worker1 # [ 187.296115] nix-daemon[847]: reaped child process 1977, status = succeeded lb # [ 187.237615] niks3-server[1098]: 2026/09/22 11:28:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 187.308738] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.919Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 187.310244] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.919Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 187.264722] niks3-server[1098]: 2026/09/22 11:28:05 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 187.341544] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.952Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.279079] niks3-server[1098]: 2026/09/22 11:28:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 187.292431] niks3-server[1098]: 2026/09/22 11:28:05 INFO Signed narinfos id=66 count=1 lb # [ 187.300793] niks3-server[1098]: 2026/09/22 11:28:05 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls lb # [ 187.312340] niks3-server[1098]: 2026/09/22 11:28:05 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete worker1 # [ 187.382721] nix-grpc-daemon[1583]: time=2026-09-22T11:28:05.993Z level=INFO msg="Upload complete. (344ms)" lb # [ 187.322051] niks3-server[1098]: 2026/09/22 11:28:05 INFO Completed upload id=66 lb # [ 187.326651] niks3-server[1098]: 2026/09/22 11:28:05 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 187.391034] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 187.406801] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 187.415248] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 187.420785] nix-daemon[847]: reaped child process 1967, status = succeeded worker1 # [ 187.433704] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 188.913009] nix-daemon[847]: reaped child process 1981, status = succeeded worker1 # [ 189.020381] nix-daemon[847]: accepted connection from pid 1997, user nix-grpc-daemon (trusted) worker1 # [ 189.036801] nix-daemon[847]: reaped child process 2000, status = succeeded lb # [ 188.981051] niks3-server[1098]: 2026/09/22 11:28:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.055904] nix-grpc-daemon[1583]: time=2026-09-22T11:28:07.664Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 189.057970] nix-grpc-daemon[1583]: time=2026-09-22T11:28:07.664Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 189.019813] niks3-server[1098]: 2026/09/22 11:28:07 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 189.033131] niks3-server[1098]: 2026/09/22 11:28:07 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker1 # [ 189.109599] nix-grpc-daemon[1583]: time=2026-09-22T11:28:07.720Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.048530] niks3-server[1098]: 2026/09/22 11:28:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 189.059080] niks3-server[1098]: 2026/09/22 11:28:07 INFO Signed narinfos id=67 count=1 lb # [ 189.064767] niks3-server[1098]: 2026/09/22 11:28:07 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 189.075207] niks3-server[1098]: 2026/09/22 11:28:07 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete worker1 # [ 189.137295] nix-grpc-daemon[1583]: time=2026-09-22T11:28:07.748Z level=INFO msg="Upload complete. (290ms)" lb # [ 189.080601] niks3-server[1098]: 2026/09/22 11:28:07 INFO Completed upload id=67 lb # [ 189.083213] niks3-server[1098]: 2026/09/22 11:28:07 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 189.151629] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=1 worker1 # [ 189.168418] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:07Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 189.174305] nix-daemon[847]: reaped child process 1979, status = succeeded worker1 # [ 189.189464] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 189.196835] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:07Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 189.207628] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 189.221808] nix-grpc-daemon[1583]: time=2026-09-22T11:28:07.832Z level=INFO msg="All 2 paths already cached" worker1 # [ 189.230551] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 189.243977] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 189.390878] nix-daemon[847]: accepted connection from pid 2006, user nix-grpc-daemon (trusted) worker1 # [ 189.411833] nix-daemon[847]: reaped child process 2009, status = succeeded worker1 # [ 189.510038] nix-daemon[847]: accepted connection from pid 2011, user nix-grpc-daemon (trusted) worker1 # [ 189.528135] nix-daemon[847]: reaped child process 2014, status = succeeded lb # [ 189.468300] niks3-server[1098]: 2026/09/22 11:28:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.539353] nix-grpc-daemon[1583]: time=2026-09-22T11:28:08.150Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 189.540990] nix-grpc-daemon[1583]: time=2026-09-22T11:28:08.150Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 189.493837] niks3-server[1098]: 2026/09/22 11:28:08 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 189.577943] nix-grpc-daemon[1583]: time=2026-09-22T11:28:08.188Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.508270] niks3-server[1098]: 2026/09/22 11:28:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 189.521420] niks3-server[1098]: 2026/09/22 11:28:08 INFO Signed narinfos id=68 count=1 lb # [ 189.527543] niks3-server[1098]: 2026/09/22 11:28:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls lb # [ 189.536929] niks3-server[1098]: 2026/09/22 11:28:08 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 189.543802] niks3-server[1098]: 2026/09/22 11:28:08 INFO Completed upload id=68 worker1 # [ 189.605478] nix-grpc-daemon[1583]: time=2026-09-22T11:28:08.216Z level=INFO msg="Upload complete. (347ms)" lb # [ 189.546080] niks3-server[1098]: 2026/09/22 11:28:08 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 189.615481] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 189.626493] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 189.635019] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 189.637479] nix-daemon[847]: reaped child process 2004, status = succeeded worker1 # [ 189.651213] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 190.520779] nix-daemon[847]: reaped child process 2018, status = succeeded worker1 # [ 190.625852] nix-daemon[847]: accepted connection from pid 2034, user nix-grpc-daemon (trusted) worker1 # [ 190.645974] nix-daemon[847]: reaped child process 2037, status = succeeded lb # [ 190.585251] niks3-server[1098]: 2026/09/22 11:28:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 190.658305] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.268Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 190.659974] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.268Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 190.621379] niks3-server[1098]: 2026/09/22 11:28:09 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 190.633414] niks3-server[1098]: 2026/09/22 11:28:09 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst worker1 # [ 190.708674] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.319Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.648177] niks3-server[1098]: 2026/09/22 11:28:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 190.657756] niks3-server[1098]: 2026/09/22 11:28:09 INFO Signed narinfos id=69 count=1 lb # [ 190.664748] niks3-server[1098]: 2026/09/22 11:28:09 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls worker1 # [ 190.730585] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.341Z level=INFO msg="Upload complete. (269ms)" lb # [ 190.678021] niks3-server[1098]: 2026/09/22 11:28:09 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 190.682984] niks3-server[1098]: 2026/09/22 11:28:09 INFO Completed upload id=69 lb # [ 190.686023] niks3-server[1098]: 2026/09/22 11:28:09 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker1 # [ 190.745127] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=1 worker1 # [ 190.760775] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:09Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 190.765588] nix-daemon[847]: reaped child process 2016, status = succeeded worker1 # [ 190.786560] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 190.801281] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 190.815039] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.426Z level=INFO msg="All 2 paths already cached" worker1 # [ 190.823985] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 190.837492] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 190.995654] nix-daemon[847]: accepted connection from pid 2041, user nix-grpc-daemon (trusted) worker1 # [ 191.011828] nix-daemon[847]: reaped child process 2044, status = succeeded worker1 # [ 191.113601] nix-daemon[847]: accepted connection from pid 2046, user nix-grpc-daemon (trusted) worker1 # [ 191.131854] nix-daemon[847]: reaped child process 2049, status = succeeded lb # [ 191.070439] niks3-server[1098]: 2026/09/22 11:28:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.141513] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.752Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 191.143028] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.752Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 191.097421] niks3-server[1098]: 2026/09/22 11:28:09 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 191.175365] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.786Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.112584] niks3-server[1098]: 2026/09/22 11:28:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 191.122268] niks3-server[1098]: 2026/09/22 11:28:09 INFO Signed narinfos id=70 count=1 lb # [ 191.129855] niks3-server[1098]: 2026/09/22 11:28:09 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 191.143140] niks3-server[1098]: 2026/09/22 11:28:09 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete worker1 # [ 191.212873] nix-grpc-daemon[1583]: time=2026-09-22T11:28:09.823Z level=INFO msg="Upload complete. (359ms)" lb # [ 191.151539] niks3-server[1098]: 2026/09/22 11:28:09 INFO Completed upload id=70 worker1 # [ 191.219044] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 191.157112] niks3-server[1098]: 2026/09/22 11:28:09 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 191.228852] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 191.239207] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 191.243987] nix-daemon[847]: reaped child process 2039, status = succeeded worker1 # [ 191.257546] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 192.359652] nix-daemon[847]: reaped child process 2053, status = succeeded worker1 # [ 192.431341] nix-daemon[847]: accepted connection from pid 2069, user nix-grpc-daemon (trusted) worker1 # [ 192.448350] nix-daemon[847]: reaped child process 2072, status = succeeded lb # [ 192.394638] niks3-server[1098]: 2026/09/22 11:28:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.467999] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.077Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 192.469844] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.077Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 192.425458] niks3-server[1098]: 2026/09/22 11:28:11 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 192.440578] niks3-server[1098]: 2026/09/22 11:28:11 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 192.452814] niks3-server[1098]: 2026/09/22 11:28:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign worker1 # [ 192.520624] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.131Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.460805] niks3-server[1098]: 2026/09/22 11:28:11 INFO Signed narinfos id=71 count=1 lb # [ 192.466562] niks3-server[1098]: 2026/09/22 11:28:11 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 192.475568] niks3-server[1098]: 2026/09/22 11:28:11 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 192.482738] niks3-server[1098]: 2026/09/22 11:28:11 INFO Completed upload id=71 worker1 # [ 192.544352] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.155Z level=INFO msg="Upload complete. (276ms)" lb # [ 192.485023] niks3-server[1098]: 2026/09/22 11:28:11 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 192.557118] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=1 worker1 # [ 192.574180] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:11Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 192.579171] nix-daemon[847]: reaped child process 2051, status = succeeded worker1 # [ 192.598483] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 192.612847] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 192.625827] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.236Z level=INFO msg="All 2 paths already cached" worker1 # [ 192.634175] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 192.646406] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 192.795032] nix-daemon[847]: accepted connection from pid 2076, user nix-grpc-daemon (trusted) worker1 # [ 192.811808] nix-daemon[847]: reaped child process 2079, status = succeeded worker1 # [ 192.913239] nix-daemon[847]: accepted connection from pid 2081, user nix-grpc-daemon (trusted) worker1 # [ 192.929937] nix-daemon[847]: reaped child process 2084, status = succeeded lb # [ 192.874242] niks3-server[1098]: 2026/09/22 11:28:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.945510] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.556Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 192.946985] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.556Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" worker1 # [ 192.977515] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.588Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.904676] niks3-server[1098]: 2026/09/22 11:28:11 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst lb # [ 192.919707] niks3-server[1098]: 2026/09/22 11:28:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 192.930416] niks3-server[1098]: 2026/09/22 11:28:11 INFO Signed narinfos id=72 count=1 lb # [ 192.936806] niks3-server[1098]: 2026/09/22 11:28:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 192.947494] niks3-server[1098]: 2026/09/22 11:28:11 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 192.953730] niks3-server[1098]: 2026/09/22 11:28:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo lb # [ 192.958877] niks3-server[1098]: 2026/09/22 11:28:11 INFO Completed upload id=72 worker1 # [ 193.021560] nix-grpc-daemon[1583]: time=2026-09-22T11:28:11.632Z level=INFO msg="Upload complete. (360ms)" worker1 # [ 193.028276] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 193.044056] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 193.052315] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 193.055202] nix-daemon[847]: reaped child process 2074, status = succeeded worker1 # [ 193.067840] nix-daemon[847]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker1 # [ 194.132711] nix-daemon[847]: reaped child process 2088, status = succeeded worker1 # [ 194.231785] nix-daemon[847]: accepted connection from pid 2104, user nix-grpc-daemon (trusted) worker1 # [ 194.252311] nix-daemon[847]: reaped child process 2107, status = succeeded lb # [ 194.189733] niks3-server[1098]: 2026/09/22 11:28:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.262638] nix-grpc-daemon[1583]: time=2026-09-22T11:28:12.872Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 194.264199] nix-grpc-daemon[1583]: time=2026-09-22T11:28:12.872Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 194.231199] niks3-server[1098]: 2026/09/22 11:28:12 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 194.307684] nix-grpc-daemon[1583]: time=2026-09-22T11:28:12.918Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.246189] niks3-server[1098]: 2026/09/22 11:28:12 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 194.256588] niks3-server[1098]: 2026/09/22 11:28:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 194.264253] niks3-server[1098]: 2026/09/22 11:28:12 INFO Signed narinfos id=73 count=1 lb # [ 194.266730] niks3-server[1098]: 2026/09/22 11:28:12 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 194.276122] niks3-server[1098]: 2026/09/22 11:28:12 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 194.280812] niks3-server[1098]: 2026/09/22 11:28:12 INFO Completed upload id=73 lb # [ 194.283427] niks3-server[1098]: 2026/09/22 11:28:12 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 194.343196] nix-grpc-daemon[1583]: time=2026-09-22T11:28:12.954Z level=INFO msg="Upload complete. (265ms)" worker1 # [ 194.355516] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46094 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=1 worker1 # [ 194.375043] nix-daemon[847]: reaped child process 2086, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.33 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 194.670580] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 8701938f9c2f4fa888d5d2d8758060db client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 194.882591] nix[1533]: don't know how to build these paths: worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client # [ 194.883939] nix[1533]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2 # [ 194.738581] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:51912 worker1 # [ 194.805193] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 client # [ 194.910020] nix[1533]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker worker1 # [ 194.826558] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:13Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 198.560880] envoy[908]: conn peer=[2001:db8:1::1]:50080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27101 tx=11348 ms=18056 client # [ 224.915667] nix[1533]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu} client # [ 224.932155] nix[1533]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 224.968202] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 224.976334] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 224.979894] systemd[1]: gpu.service: Consumed 209ms CPU time over 30.303s wall clock time, 37.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.12 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.19 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1 # [ 226.465513] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:45Z level=info event=unhealthy reason=min_free worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.18 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.03 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 worker1 # [ 226.764594] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:46094 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker1 # [ 226.776935] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:46094 worker1 # [ 226.804112] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:45Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker1 # [ 226.810917] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:45Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 226.762150] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 226.781856] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51912 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 226.799056] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 226.818240] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 226.844751] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:45Z level=info event=niks3_push_started pid=2046 worker2 # [ 226.993154] nix-grpc-daemon[2046]: time=2026-09-22T11:28:45.607Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 227.098700] nix-daemon[836]: accepted connection from pid 2058, user nix-grpc-daemon (trusted) worker2 # [ 227.121341] nix-daemon[836]: reaped child process 2061, status = succeeded worker2 # [ 227.215373] nix-daemon[836]: accepted connection from pid 2063, user nix-grpc-daemon (trusted) worker2 # [ 227.232170] nix-daemon[836]: reaped child process 2066, status = succeeded lb # [ 227.238999] niks3-server[1098]: 2026/09/22 11:28:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 227.252229] nix-grpc-daemon[2046]: time=2026-09-22T11:28:45.867Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 227.255442] nix-grpc-daemon[2046]: time=2026-09-22T11:28:45.868Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 227.282157] niks3-server[1098]: 2026/09/22 11:28:45 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 227.308451] nix-grpc-daemon[2046]: time=2026-09-22T11:28:45.922Z level=INFO msg="Uploading 1 narinfos" lb # [ 227.306375] niks3-server[1098]: 2026/09/22 11:28:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 227.316159] niks3-server[1098]: 2026/09/22 11:28:45 INFO Signed narinfos id=74 count=1 lb # [ 227.318533] niks3-server[1098]: 2026/09/22 11:28:45 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 227.340240] niks3-server[1098]: 2026/09/22 11:28:45 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 227.347893] niks3-server[1098]: 2026/09/22 11:28:45 INFO Completed upload id=74 worker2 # [ 227.349645] nix-grpc-daemon[2046]: time=2026-09-22T11:28:45.964Z level=INFO msg="Upload complete. (352ms)" lb # [ 227.352528] niks3-server[1098]: 2026/09/22 11:28:45 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 227.360410] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51912 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 227.379840] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 227.392412] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 227.395885] nix-daemon[836]: reaped child process 2044, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 227.411534] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 228.236479] nix-daemon[836]: reaped child process 2070, status = succeeded worker2 # [ 228.333715] nix-daemon[836]: accepted connection from pid 2086, user nix-grpc-daemon (trusted) worker2 # [ 228.353783] nix-daemon[836]: reaped child process 2089, status = succeeded lb # [ 228.351768] niks3-server[1098]: 2026/09/22 11:28:46 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 228.368328] nix-grpc-daemon[2046]: time=2026-09-22T11:28:46.980Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 228.369752] nix-grpc-daemon[2046]: time=2026-09-22T11:28:46.980Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 228.387633] niks3-server[1098]: 2026/09/22 11:28:46 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 228.405960] niks3-server[1098]: 2026/09/22 11:28:47 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 228.419483] niks3-server[1098]: 2026/09/22 11:28:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign worker2 # [ 228.425857] nix-grpc-daemon[2046]: time=2026-09-22T11:28:47.039Z level=INFO msg="Uploading 1 narinfos" lb # [ 228.427899] niks3-server[1098]: 2026/09/22 11:28:47 INFO Signed narinfos id=75 count=1 lb # [ 228.433440] niks3-server[1098]: 2026/09/22 11:28:47 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 228.444797] niks3-server[1098]: 2026/09/22 11:28:47 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 228.450623] niks3-server[1098]: 2026/09/22 11:28:47 INFO Completed upload id=75 worker2 # [ 228.452196] nix-grpc-daemon[2046]: time=2026-09-22T11:28:47.066Z level=INFO msg="Upload complete. (231ms)" lb # [ 228.454309] niks3-server[1098]: 2026/09/22 11:28:47 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 228.464876] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=1 worker1 # [ 228.530745] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:47Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker1 # [ 228.536709] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:47Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 228.479918] nix-daemon[836]: reaped child process 2068, status = succeeded worker2 # [ 228.514564] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 228.530875] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 228.545230] nix-grpc-daemon[2046]: time=2026-09-22T11:28:47.159Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 228.555461] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51912 lb # [ 228.559164] envoy[908]: conn peer=[2001:db8:1::1]:54284 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=30059 worker2 # [ 228.568670] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 228.724507] nix-daemon[836]: accepted connection from pid 2093, user nix-grpc-daemon (trusted) worker2 # [ 228.741369] nix-daemon[836]: reaped child process 2096, status = succeeded worker2 # [ 228.839108] nix-daemon[836]: accepted connection from pid 2098, user nix-grpc-daemon (trusted) worker2 # [ 228.853303] nix-daemon[836]: reaped child process 2101, status = succeeded lb # [ 228.859439] niks3-server[1098]: 2026/09/22 11:28:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 228.869452] nix-grpc-daemon[2046]: time=2026-09-22T11:28:47.483Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 228.870981] nix-grpc-daemon[2046]: time=2026-09-22T11:28:47.483Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 228.886350] niks3-server[1098]: 2026/09/22 11:28:47 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 228.903314] nix-grpc-daemon[2046]: time=2026-09-22T11:28:47.517Z level=INFO msg="Uploading 1 narinfos" lb # [ 228.901988] niks3-server[1098]: 2026/09/22 11:28:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign lb # [ 228.911784] niks3-server[1098]: 2026/09/22 11:28:47 INFO Signed narinfos id=76 count=1 lb # [ 228.917090] niks3-server[1098]: 2026/09/22 11:28:47 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 228.930129] niks3-server[1098]: 2026/09/22 11:28:47 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete worker2 # [ 228.936346] nix-grpc-daemon[2046]: time=2026-09-22T11:28:47.551Z level=INFO msg="Upload complete. (351ms)" lb # [ 228.938122] niks3-server[1098]: 2026/09/22 11:28:47 INFO Completed upload id=76 lb # [ 228.943519] niks3-server[1098]: 2026/09/22 11:28:47 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 228.944353] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51912 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 228.957982] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 228.964250] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 228.969111] nix-daemon[836]: reaped child process 2091, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 228.981452] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 230.034669] nix-daemon[836]: reaped child process 2105, status = succeeded worker2 # [ 230.145781] nix-daemon[836]: accepted connection from pid 2121, user nix-grpc-daemon (trusted) worker2 # [ 230.165313] nix-daemon[836]: reaped child process 2124, status = succeeded lb # [ 230.170637] niks3-server[1098]: 2026/09/22 11:28:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 230.183578] nix-grpc-daemon[2046]: time=2026-09-22T11:28:48.797Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 230.185544] nix-grpc-daemon[2046]: time=2026-09-22T11:28:48.797Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 230.203207] niks3-server[1098]: 2026/09/22 11:28:48 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 230.217959] niks3-server[1098]: 2026/09/22 11:28:48 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 230.232114] niks3-server[1098]: 2026/09/22 11:28:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker2 # [ 230.240384] nix-grpc-daemon[2046]: time=2026-09-22T11:28:48.854Z level=INFO msg="Uploading 1 narinfos" lb # [ 230.242269] niks3-server[1098]: 2026/09/22 11:28:48 INFO Signed narinfos id=77 count=1 lb # [ 230.249600] niks3-server[1098]: 2026/09/22 11:28:48 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 230.259547] niks3-server[1098]: 2026/09/22 11:28:48 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 230.267533] niks3-server[1098]: 2026/09/22 11:28:48 INFO Completed upload id=77 worker2 # [ 230.269192] nix-grpc-daemon[2046]: time=2026-09-22T11:28:48.883Z level=INFO msg="Upload complete. (277ms)" lb # [ 230.274668] niks3-server[1098]: 2026/09/22 11:28:48 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 230.281285] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=1 worker1 # [ 230.351110] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:48Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker1 # [ 230.357211] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:48Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 230.302010] nix-daemon[836]: reaped child process 2103, status = succeeded worker2 # [ 230.334469] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 230.350047] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 230.364558] nix-grpc-daemon[2046]: time=2026-09-22T11:28:48.979Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 230.375731] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 230.389423] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 230.541857] nix-daemon[836]: accepted connection from pid 2128, user nix-grpc-daemon (trusted) worker2 # [ 230.564303] nix-daemon[836]: reaped child process 2131, status = succeeded worker2 # [ 230.660630] nix-daemon[836]: accepted connection from pid 2133, user nix-grpc-daemon (trusted) worker2 # [ 230.681293] nix-daemon[836]: reaped child process 2136, status = succeeded lb # [ 230.677771] niks3-server[1098]: 2026/09/22 11:28:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 230.687626] nix-grpc-daemon[2046]: time=2026-09-22T11:28:49.302Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 230.689761] nix-grpc-daemon[2046]: time=2026-09-22T11:28:49.304Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 230.706038] niks3-server[1098]: 2026/09/22 11:28:49 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 230.721293] nix-grpc-daemon[2046]: time=2026-09-22T11:28:49.335Z level=INFO msg="Uploading 1 narinfos" lb # [ 230.720750] niks3-server[1098]: 2026/09/22 11:28:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign lb # [ 230.732570] niks3-server[1098]: 2026/09/22 11:28:49 INFO Signed narinfos id=78 count=1 lb # [ 230.738566] niks3-server[1098]: 2026/09/22 11:28:49 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls lb # [ 230.750193] niks3-server[1098]: 2026/09/22 11:28:49 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete worker2 # [ 230.757311] nix-grpc-daemon[2046]: time=2026-09-22T11:28:49.372Z level=INFO msg="Upload complete. (351ms)" lb # [ 230.756144] niks3-server[1098]: 2026/09/22 11:28:49 INFO Completed upload id=78 worker2 # [ 230.765104] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51912 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 230.762142] niks3-server[1098]: 2026/09/22 11:28:49 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 230.776904] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 worker2 # [ 230.786119] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 230.791321] nix-daemon[836]: reaped child process 2126, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 230.798445] nix-daemon[836]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 231.635101] nix-daemon[836]: reaped child process 2140, status = succeeded worker2 # [ 231.752101] nix-daemon[836]: accepted connection from pid 2156, user nix-grpc-daemon (trusted) worker2 # [ 231.769858] nix-daemon[836]: reaped child process 2159, status = succeeded lb # [ 231.775611] niks3-server[1098]: 2026/09/22 11:28:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 231.789484] nix-grpc-daemon[2046]: time=2026-09-22T11:28:50.402Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 231.791057] nix-grpc-daemon[2046]: time=2026-09-22T11:28:50.402Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 231.811015] niks3-server[1098]: 2026/09/22 11:28:50 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 231.831402] nix-grpc-daemon[2046]: time=2026-09-22T11:28:50.445Z level=INFO msg="Uploading 1 narinfos" lb # [ 231.829442] niks3-server[1098]: 2026/09/22 11:28:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 231.843055] niks3-server[1098]: 2026/09/22 11:28:50 INFO Signed narinfos id=79 count=1 lb # [ 231.849055] niks3-server[1098]: 2026/09/22 11:28:50 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 231.858205] niks3-server[1098]: 2026/09/22 11:28:50 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 231.869283] niks3-server[1098]: 2026/09/22 11:28:50 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete worker2 # [ 231.870314] nix-grpc-daemon[2046]: time=2026-09-22T11:28:50.485Z level=INFO msg="Upload complete. (261ms)" lb # [ 231.874812] niks3-server[1098]: 2026/09/22 11:28:50 INFO Completed upload id=79 lb # [ 231.878246] niks3-server[1098]: 2026/09/22 11:28:50 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 231.885734] nix-grpc-daemon[1986]: ts=2026-09-22T11:28:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51912 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=1 worker2 # [ 231.902021] nix-daemon[836]: reaped child process 2138, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.46 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 232.467401] nix-grpc-daemon[1487]: ts=2026-09-22T11:28:51Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 8.20 seconds) (finished: run the VM test script, in 234.36 seconds) test script finished in 234.61s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-22T11:28:52Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T11:28:52Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T11:28:52Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T11:28:53Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.41 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh