tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm client # Disk image does not exist, creating the virtualisation disk image... client # Formatting '/build/vm-state-client/tmp.u2TIzZtfXI', fmt=raw size=1073741824 worker2 # Disk image does not exist, creating the virtualisation disk image... worker2: QEMU running (pid 51) worker1 # Disk image does not exist, creating the virtualisation disk image... lb # Disk image does not exist, creating the virtualisation disk image... lb: QEMU running (pid 50) client # mke2fs 1.47.4 (6-Mar-2025) worker1 # Formatting '/build/vm-state-worker1/tmp.KKEGgw2m3y', fmt=raw size=1073741824 lb # Formatting '/build/vm-state-lb/tmp.qT35F1tNkD', fmt=raw size=1073741824 client: QEMU running (pid 49) worker1: QEMU running (pid 52) worker2 # Formatting '/build/vm-state-worker2/tmp.QFHS1GhFKd', fmt=raw size=1073741824 client # Discarding device blocks: 0/262144 done lb # mke2fs 1.47.4 (6-Mar-2025) worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # mke2fs 1.47.4 (6-Mar-2025) client # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Discarding device blocks: 0/262144 done worker2 # Discarding device blocks: 0/262144 done client # Filesystem UUID: ae37c38b-e469-4e07-87a2-8424a7829175 worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client # Superblock backups stored on blocks: worker1 # Filesystem UUID: 933f64f8-b512-4fd9-b502-2906bfb81b78 worker2 # Filesystem UUID: db3a8e93-7bbf-4dc9-b691-45970539d6f1 client # 32768, 98304, 163840, 229376 worker1 # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: client # worker1 # 32768, 98304, 163840, 229376 worker2 # 32768, 98304, 163840, 229376 client # Allocating group tables: 0/8 done worker1 # worker2 # client # Writing inode tables: 0/8 done worker1 # Allocating group tables: 0/8 done worker2 # Allocating group tables: 0/8 done client # Creating journal (8192 blocks): done worker1 # Writing inode tables: 0/8 done worker2 # Writing inode tables: 0/8 done client # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Creating journal (8192 blocks): done worker2 # Creating journal (8192 blocks): done client # worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # Virtualisation disk image created. worker1 # worker2 # client # Starting virtiofs daemons... worker1 # Virtualisation disk image created. worker2 # Virtualisation disk image created. client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # Starting virtiofs daemons... worker2 # Starting virtiofs daemons... client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... (finished: start all VMs, in 0.50 seconds) lb # Discarding device blocks: 0/262144 done client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client: waiting for unit network-addresses-eth1.service worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client: waiting for the VM to finish booting worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Creating filesystem with 262144 4k blocks and 65536 inodes client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # Filesystem UUID: 4e796412-306a-4bfb-ae60-e56a025ef008 client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # Superblock backups stored on blocks: client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # 32768, 98304, 163840, 229376 client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # lb # Allocating group tables: 0/8 done lb # Writing inode tables: 0/8 done lb # Creating journal (8192 blocks): done lb # Writing superblocks and filesystem accounting information: 0/8 done lb # lb # Virtualisation disk image created. lb # Starting virtiofs daemons... lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] KASLR enabled lb # [ 0.000000] random: crng init done lb # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] Zone ranges: lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] DMA32 empty lb # [ 0.000000] Normal empty lb # [ 0.000000] Device empty lb # [ 0.000000] Movable zone start for each node lb # [ 0.000000] Early memory node ranges lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 lb # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3in2qynk4v72bmzzvgahb746q83gs855-nixos-system-lb-test/init regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration", will be passed to user space. lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] Policy zone: DMA lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.000036] arm-pv: using stolen time PV lb # [ 0.000661] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] lb # [ 0.000805] Console: colour dummy device 80x25 lb # [ 0.000813] printk: legacy console [tty0] enabled worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] KASLR enabled lb # [ 0.001002] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000000] random: crng init done lb # [ 0.001009] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] efi: UEFI not found. lb # [ 0.001094] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.001279] landlock: Up and running. worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.001282] Yama: becoming mindful. lb # [ 0.001951] LSM support for eBPF active worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.002124] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Zone ranges: lb # [ 0.002156] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] DMA32 empty lb # [ 0.004027] rcu: Hierarchical SRCU implementation. worker1 # [ 0.000000] Normal empty lb # [ 0.004032] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.000000] Device empty lb # [ 0.005258] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] Early memory node ranges lb # [ 0.005356] EFI services will not be available. lb # [ 0.005480] smp: Bringing up secondary CPUs ... worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005489] smp: Brought up 1 node, 1 CPU worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005492] SMP: Total of 1 processors activated. lb # [ 0.005497] CPU: All CPU(s) started at EL1 worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.005507] CPU features: detected: Branch Target Identification worker2 # [ 0.000000] KASLR enabled worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] random: crng init done lb # [ 0.005511] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] efi: UEFI not found. lb # [ 0.005515] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.005518] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.005522] CPU features: detected: Common not Private translations client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] KASLR enabled worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.005525] CPU features: detected: CRC32 instructions worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] Zone ranges: client # [ 0.000000] random: crng init done worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005528] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] DMA32 empty worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] Normal empty lb # [ 0.005531] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.000000] Device empty client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker2 # [ 0.000000] Movable zone start for each node lb # [ 0.005535] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] Early memory node ranges client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005538] CPU features: detected: E0PD worker1 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005540] CPU features: detected: Enhanced Counter Virtualization client # [ 0.000000] Zone ranges: worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005543] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] DMA32 empty worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 lb # [ 0.005547] CPU features: detected: Enhanced Virtualization Traps client # [ 0.000000] Normal empty worker1 # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] Device empty lb # [ 0.005549] CPU features: detected: Fine Grained Traps client # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] Early memory node ranges lb # [ 0.005553] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.005557] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.005560] CPU features: detected: LSE atomic instructions client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.005562] CPU features: detected: Privileged Access Never client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/waj0blgjl9bpqbg9b6xiyknfyf8i56g7-nixos-system-worker1-test/init regInfo=/nix/store/036mdri2akx3g3s2sm0y9jgc449gc7mb-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.005565] CPU features: detected: PMUv3 client # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.005568] CPU features: detected: RAS Extension Support client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.005571] CPU features: detected: RASv1p1 Extension Support client # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.005573] CPU features: detected: Random Number Generator worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/036mdri2akx3g3s2sm0y9jgc449gc7mb-closure-info/registration", will be passed to user space. client # [ 0.000000] psci: SMC Calling Convention v1.1 worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.005576] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.005578] CPU features: detected: Stage-2 Force Write-Back client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.005581] CPU features: detected: TLB range maintenance instructions client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.005586] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.005626] alternatives: applying system-wide alternatives client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.008507] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.008664] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.009035] devtmpfs: initialized worker1 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yhybdyymi2isy70630z4y9gwglld4pzg-nixos-system-worker2-test/init regInfo=/nix/store/p2vhx3y5597gdgv9d5wglk0drd0sdzdn-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.010889] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] Policy zone: DMA lb # [ 0.010911] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.011094] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/p2vhx3y5597gdgv9d5wglk0drd0sdzdn-closure-info/registration", will be passed to user space. worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.011098] 0 pages in range for non-PLT usage worker1 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.011099] 508272 pages in range for PLT usage worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.011187] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.012005] DMI not present or invalid. client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test/init regInfo=/nix/store/x4pxw657nkinqgkz3xzfplm1jf2xbslh-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.015663] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.017833] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x4pxw657nkinqgkz3xzfplm1jf2xbslh-closure-info/registration", will be passed to user space. worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.018008] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.018209] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.018230] audit: initializing netlink subsys (disabled) worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.018828] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.018830] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.018834] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.018837] thermal_sys: Registered thermal governor 'user_space' client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.018842] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 client # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.018867] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.018875] cpuidle: using governor ladder worker2 # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.018881] cpuidle: using governor menu client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.019079] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.019095] ASID allocator initialised with 65536 entries client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.020348] Serial: AMBA PL011 UART driver client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.025892] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.026049] printk: console [ttyAMA0] enabled client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.143779] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.143794] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.143800] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.143804] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.143808] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.000033] arm-pv: using stolen time PV client # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.143813] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.143817] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.000522] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000693] Console: colour dummy device 80x25 lb # [ 0.143821] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000700] printk: legacy console [tty0] enabled client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000887] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000893] pid_max: default: 32768 minimum: 301 lb # [ 0.150962] fbcon: Taking over console lb # [ 0.150975] ACPI: Interpreter disabled. client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000964] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.001123] landlock: Up and running. client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.001126] Yama: becoming mindful. worker1 # [ 0.001655] LSM support for eBPF active client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.001805] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.001846] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000028] arm-pv: using stolen time PV worker1 # [ 0.003665] rcu: Hierarchical SRCU implementation. client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.003671] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.000370] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.004913] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000593] Console: colour dummy device 80x25 client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.005004] EFI services will not be available. worker2 # [ 0.000602] printk: legacy console [tty0] enabled worker1 # [ 0.005117] smp: Bringing up secondary CPUs ... client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.005125] smp: Brought up 1 node, 1 CPU client # [ 0.000029] arm-pv: using stolen time PV worker2 # [ 0.000816] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.005128] SMP: Total of 1 processors activated. worker1 # [ 0.005133] CPU: All CPU(s) started at EL1 worker2 # [ 0.000823] pid_max: default: 32768 minimum: 301 client # [ 0.000406] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.159926] iommu: Default domain type: Translated worker2 # [ 0.000897] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.005143] CPU features: detected: Branch Target Identification client # [ 0.000572] Console: colour dummy device 80x25 lb # [ 0.159936] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.001014] landlock: Up and running. client # [ 0.000580] printk: legacy console [tty0] enabled worker1 # [ 0.005148] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.160278] SCSI subsystem initialized worker2 # [ 0.001016] Yama: becoming mindful. worker2 # [ 0.001435] LSM support for eBPF active worker1 # [ 0.005151] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.000772] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.001582] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.162166] usbcore: registered new interface driver usbfs client # [ 0.000780] pid_max: default: 32768 minimum: 301 lb # [ 0.162210] usbcore: registered new interface driver hub worker1 # [ 0.005154] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.001615] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.000851] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.162226] usbcore: registered new device driver usb worker1 # [ 0.005158] CPU features: detected: Common not Private translations worker2 # [ 0.003474] rcu: Hierarchical SRCU implementation. client # [ 0.000981] landlock: Up and running. lb # [ 0.162490] pps_core: LinuxPPS API ver. 1 registered client # [ 0.000984] Yama: becoming mindful. worker1 # [ 0.005161] CPU features: detected: CRC32 instructions worker2 # [ 0.003479] rcu: Max phase no-delay instances is 1000. client # [ 0.001413] LSM support for eBPF active worker2 # [ 0.004773] fsl-mc MSI: its@8080000 domain created lb # [ 0.162496] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.005164] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.001563] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.004863] EFI services will not be available. lb # [ 0.162507] PTP clock support registered lb # [ 0.162558] EDAC MC: Ver: 3.0.0 worker1 # [ 0.005168] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.004927] smp: Bringing up secondary CPUs ... client # [ 0.001595] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.004936] smp: Brought up 1 node, 1 CPU lb # [ 0.166965] scmi_core: SCMI protocol bus registered worker1 # [ 0.005170] CPU features: detected: Data independent timing control (DIT) client # [ 0.003362] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004939] SMP: Total of 1 processors activated. worker1 # [ 0.005174] CPU features: detected: E0PD lb # [ 0.167886] FPGA manager framework worker2 # [ 0.004944] CPU: All CPU(s) started at EL1 client # [ 0.003367] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.005176] CPU features: detected: Enhanced Counter Virtualization client # [ 0.004613] fsl-mc MSI: its@8080000 domain created lb # [ 0.168793] vgaarb: loaded worker2 # [ 0.004954] CPU features: detected: Branch Target Identification client # [ 0.004705] EFI services will not be available. worker1 # [ 0.005179] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.004959] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.004812] smp: Bringing up secondary CPUs ... lb # [ 0.169383] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.005183] CPU features: detected: Enhanced Virtualization Traps client # [ 0.004821] smp: Brought up 1 node, 1 CPU worker2 # [ 0.004962] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.004824] SMP: Total of 1 processors activated. worker1 # [ 0.005186] CPU features: detected: Fine Grained Traps client # [ 0.004829] CPU: All CPU(s) started at EL1 worker2 # [ 0.004966] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.005190] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.004838] CPU features: detected: Branch Target Identification worker2 # [ 0.004969] CPU features: detected: Common not Private translations worker1 # [ 0.005195] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.004843] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.004973] CPU features: detected: CRC32 instructions worker1 # [ 0.005198] CPU features: detected: LSE atomic instructions client # [ 0.004846] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.004975] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.005200] CPU features: detected: Privileged Access Never worker1 # [ 0.005203] CPU features: detected: PMUv3 worker2 # [ 0.004979] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.004850] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.174586] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.005206] CPU features: detected: RAS Extension Support worker2 # [ 0.004982] CPU features: detected: Data independent timing control (DIT) client # [ 0.004854] CPU features: detected: Common not Private translations lb # [ 0.174621] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.004985] CPU features: detected: E0PD worker1 # [ 0.005209] CPU features: detected: RASv1p1 Extension Support client # [ 0.004857] CPU features: detected: CRC32 instructions worker1 # [ 0.005212] CPU features: detected: Random Number Generator worker2 # [ 0.004988] CPU features: detected: Enhanced Counter Virtualization client # [ 0.004860] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.005214] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.004991] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.004863] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.005217] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.004995] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.178162] netfs: FS-Cache loaded client # [ 0.004866] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.005220] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.004998] CPU features: detected: Fine Grained Traps lb # [ 0.178274] pnp: PnP ACPI: disabled client # [ 0.004869] CPU features: detected: E0PD worker1 # [ 0.005225] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.004872] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.005001] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.005263] alternatives: applying system-wide alternatives worker2 # [ 0.005005] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.004874] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.008194] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.005008] CPU features: detected: LSE atomic instructions client # [ 0.004878] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.005011] CPU features: detected: Privileged Access Never client # [ 0.004881] CPU features: detected: Fine Grained Traps worker2 # [ 0.005013] CPU features: detected: PMUv3 worker1 # [ 0.008379] Memory: 1405668K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) client # [ 0.004884] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.005016] CPU features: detected: RAS Extension Support client # [ 0.004888] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.005019] CPU features: detected: RASv1p1 Extension Support client # [ 0.004891] CPU features: detected: LSE atomic instructions worker2 # [ 0.005022] CPU features: detected: Random Number Generator client # [ 0.004894] CPU features: detected: Privileged Access Never worker2 # [ 0.005024] CPU features: detected: Speculation barrier (SB) client # [ 0.004897] CPU features: detected: PMUv3 worker2 # [ 0.005027] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.182127] NET: Registered PF_INET protocol family client # [ 0.004899] CPU features: detected: RAS Extension Support client # [ 0.004902] CPU features: detected: RASv1p1 Extension Support lb # [ 0.182399] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.004904] CPU features: detected: Random Number Generator client # [ 0.004907] CPU features: detected: Speculation barrier (SB) client # [ 0.004909] CPU features: detected: Stage-2 Force Write-Back client # [ 0.004912] CPU features: detected: TLB range maintenance instructions client # [ 0.004916] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.004952] alternatives: applying system-wide alternatives worker1 # [ 0.008757] devtmpfs: initialized client # [ 0.007779] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.010485] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.010506] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.007954] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) worker1 # [ 0.010682] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.008332] devtmpfs: initialized worker1 # [ 0.010687] 0 pages in range for non-PLT usage worker2 # [ 0.005030] CPU features: detected: TLB range maintenance instructions client # [ 0.010123] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.010688] 508272 pages in range for PLT usage worker2 # [ 0.005034] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.010808] pinctrl core: initialized pinctrl subsystem client # [ 0.010146] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.005070] alternatives: applying system-wide alternatives worker1 # [ 0.011653] DMI not present or invalid. client # [ 0.010340] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.007990] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.014784] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.010344] 0 pages in range for non-PLT usage client # [ 0.010345] 508272 pages in range for PLT usage worker1 # [ 0.016991] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.010435] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.008248] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) worker1 # [ 0.017181] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.011143] DMI not present or invalid. worker2 # [ 0.008637] devtmpfs: initialized client # [ 0.014144] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.017380] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.010413] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.016473] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.017401] audit: initializing netlink subsys (disabled) worker2 # [ 0.010435] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.018004] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.016658] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.010612] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.018006] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.010617] 0 pages in range for non-PLT usage client # [ 0.016848] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.018009] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.010618] 508272 pages in range for PLT usage client # [ 0.016869] audit: initializing netlink subsys (disabled) worker2 # [ 0.010713] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.018012] thermal_sys: Registered thermal governor 'user_space' client # [ 0.017458] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.011467] DMI not present or invalid. worker1 # [ 0.018017] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.017460] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.014671] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.017464] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.018049] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.016853] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.018058] cpuidle: using governor ladder client # [ 0.017466] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.018064] cpuidle: using governor menu worker2 # [ 0.017030] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.017471] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.018249] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.017228] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.017501] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.018265] ASID allocator initialised with 65536 entries worker2 # [ 0.017248] audit: initializing netlink subsys (disabled) client # [ 0.017511] cpuidle: using governor ladder worker1 # [ 0.019548] Serial: AMBA PL011 UART driver client # [ 0.017517] cpuidle: using governor menu worker2 # [ 0.017797] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.024699] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.017693] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.017800] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.024850] printk: console [ttyAMA0] enabled client # [ 0.017707] ASID allocator initialised with 65536 entries worker2 # [ 0.017804] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.018902] Serial: AMBA PL011 UART driver worker2 # [ 0.017806] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.147114] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.017811] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.147133] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.023974] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.017843] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.147139] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.024144] printk: console [ttyAMA0] enabled worker2 # [ 0.017851] cpuidle: using governor ladder worker1 # [ 0.147143] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.017858] cpuidle: using governor menu worker1 # [ 0.147148] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.018044] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.018060] ASID allocator initialised with 65536 entries worker1 # [ 0.147152] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.019224] Serial: AMBA PL011 UART driver worker1 # [ 0.147157] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker2 # [ 0.024500] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.147161] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.024619] printk: console [ttyAMA0] enabled worker2 # [ 0.144060] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.154673] fbcon: Taking over console worker2 # [ 0.144077] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.154686] ACPI: Interpreter disabled. client # [ 0.141372] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.144082] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.141386] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.144086] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.141391] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.144090] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.141395] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.144095] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.212025] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.141399] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.144099] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.141404] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.212065] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.144103] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.141408] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.212088] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.141412] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.212157] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.162531] iommu: Default domain type: Translated worker2 # [ 0.151513] fbcon: Taking over console lb # [ 0.212300] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.151525] ACPI: Interpreter disabled. worker1 # [ 0.162540] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.212371] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.153315] iommu: Default domain type: Translated client # [ 0.148727] fbcon: Taking over console worker1 # [ 0.164225] SCSI subsystem initialized lb # [ 0.212403] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.153325] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.148741] ACPI: Interpreter disabled. worker2 # [ 0.154940] SCSI subsystem initialized lb # [ 0.212438] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.164912] usbcore: registered new interface driver usbfs lb # [ 0.212538] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.164951] usbcore: registered new interface driver hub lb # [ 0.212589] NET: Registered PF_XDP protocol family worker2 # [ 0.155636] usbcore: registered new interface driver usbfs worker1 # [ 0.164967] usbcore: registered new device driver usb lb # [ 0.212604] PCI: CLS 0 bytes, default 64 worker2 # [ 0.155668] usbcore: registered new interface driver hub worker1 # [ 0.165237] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.212827] Trying to unpack rootfs image as initramfs... worker2 # [ 0.155684] usbcore: registered new device driver usb worker1 # [ 0.165243] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.155944] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.165254] PTP clock support registered worker1 # [ 0.165310] EDAC MC: Ver: 3.0.0 worker2 # [ 0.155951] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.155960] PTP clock support registered worker1 # [ 0.169967] scmi_core: SCMI protocol bus registered worker2 # [ 0.156009] EDAC MC: Ver: 3.0.0 worker1 # [ 0.170976] FPGA manager framework worker1 # [ 0.171910] vgaarb: loaded client # [ 0.157547] iommu: Default domain type: Translated worker1 # [ 0.172532] clocksource: Switched to clocksource arch_sys_counter client # [ 0.157556] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.158005] SCSI subsystem initialized lb # [ 0.230863] kvm [1]: HYP mode not available client # [ 0.159965] usbcore: registered new interface driver usbfs client # [ 0.159996] usbcore: registered new interface driver hub client # [ 0.160012] usbcore: registered new device driver usb client # [ 0.160276] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.177071] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.177106] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.160283] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.167538] scmi_core: SCMI protocol bus registered client # [ 0.160292] PTP clock support registered worker1 # [ 0.178790] netfs: FS-Cache loaded client # [ 0.160344] EDAC MC: Ver: 3.0.0 worker1 # [ 0.178914] pnp: PnP ACPI: disabled worker2 # [ 0.168520] FPGA manager framework worker2 # [ 0.169421] vgaarb: loaded client # [ 0.164956] scmi_core: SCMI protocol bus registered client # [ 0.165899] FPGA manager framework client # [ 0.166809] vgaarb: loaded worker2 # [ 0.170061] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.170648] VFS: Disk quotas dquot_6.6.0 client # [ 0.167424] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.170679] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.173059] netfs: FS-Cache loaded worker2 # [ 0.173150] pnp: PnP ACPI: disabled worker1 # [ 0.182774] NET: Registered PF_INET protocol family worker1 # [ 0.183042] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.171201] VFS: Disk quotas dquot_6.6.0 client # [ 0.171240] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.174841] netfs: FS-Cache loaded client # [ 0.174954] pnp: PnP ACPI: disabled worker2 # [ 0.178966] NET: Registered PF_INET protocol family worker2 # [ 0.179238] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.178763] NET: Registered PF_INET protocol family client # [ 0.179012] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.213085] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.213128] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.213154] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.213229] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.213371] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.208034] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.213450] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.208076] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.213486] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208102] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.213521] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208173] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.213614] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.213636] NET: Registered PF_XDP protocol family worker2 # [ 0.208317] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.213653] PCI: CLS 0 bytes, default 64 worker2 # [ 0.208403] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.213878] Trying to unpack rootfs image as initramfs... client # [ 0.208888] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.208436] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208472] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.208929] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.208593] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.208956] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.208615] NET: Registered PF_XDP protocol family worker2 # [ 0.208631] PCI: CLS 0 bytes, default 64 client # [ 0.209030] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.208866] Trying to unpack rootfs image as initramfs... client # [ 0.209172] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.209252] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.209286] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.230468] kvm [1]: HYP mode not available client # [ 0.209324] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.209449] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.209469] NET: Registered PF_XDP protocol family client # [ 0.209485] PCI: CLS 0 bytes, default 64 client # [ 0.209706] Trying to unpack rootfs image as initramfs... worker2 # [ 0.224034] kvm [1]: HYP mode not available client # [ 0.225080] kvm [1]: HYP mode not available lb # [ 0.314274] Initialise system trusted keyrings lb # [ 0.314998] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.316211] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.321412] 9p: Installing v9fs 9p2000 file system support lb # [ 0.342076] Key type asymmetric registered lb # [ 0.342093] Asymmetric key parser 'x509' registered lb # [ 0.342151] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.344210] io scheduler mq-deadline registered lb # [ 0.344221] io scheduler kyber registered lb # [ 0.353519] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.354868] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.355250] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.355267] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.355279] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.355288] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.355307] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.355330] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.355400] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.355409] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.355414] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.355419] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.355424] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.355477] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.355908] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.356089] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.356105] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.356135] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.356151] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.356601] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.356779] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.356795] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.356824] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.357264] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.321113] Initialise system trusted keyrings worker1 # [ 0.321847] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.323083] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.323889] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.314566] Initialise system trusted keyrings worker2 # [ 0.315307] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.316555] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.317341] 9p: Installing v9fs 9p2000 file system support client # [ 0.315928] Initialise system trusted keyrings lb # [ 0.385508] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.316664] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.385536] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.385566] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.317961] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.386058] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.386244] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.318732] 9p: Installing v9fs 9p2000 file system support lb # [ 0.386260] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.386290] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.386309] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.386765] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.386946] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.386975] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.387422] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.387608] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.387639] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.388062] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.388243] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.388491] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.388676] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388706] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.389149] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.389330] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.389359] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.338045] Key type asymmetric registered lb # [ 0.389831] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.338063] Asymmetric key parser 'x509' registered lb # [ 0.390016] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.390046] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.390502] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.390760] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.352591] Key type asymmetric registered lb # [ 0.390777] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.352614] Asymmetric key parser 'x509' registered lb # [ 0.390806] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.339407] Key type asymmetric registered worker1 # [ 0.352681] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.339423] Asymmetric key parser 'x509' registered lb # [ 0.391252] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.354796] io scheduler mq-deadline registered lb # [ 0.391433] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.354806] io scheduler kyber registered lb # [ 0.391449] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.391478] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.346181] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.392036] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.347230] io scheduler mq-deadline registered worker2 # [ 0.347241] io scheduler kyber registered lb # [ 0.392047] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.392052] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.392095] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.392140] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.392186] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.347516] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.348549] io scheduler mq-deadline registered lb # [ 0.392231] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.348559] io scheduler kyber registered lb # [ 0.392276] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.392321] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.364671] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.392366] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.392410] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.392455] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.366058] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.392521] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.358231] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.366445] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.392565] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.366461] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.392587] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.392608] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.366474] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.392629] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.359552] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.366482] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.392650] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.359939] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.366508] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.392671] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.359956] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.392692] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.366530] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.359982] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.392716] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.366612] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.392737] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.366621] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.359991] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.392759] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.366627] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.360017] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.392781] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.366632] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.360040] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.392803] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.366637] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.360112] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.392824] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.366697] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.360121] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.392845] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.360127] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.367144] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.392866] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.367329] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.360133] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.392886] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned client # [ 0.359793] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.367345] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.392907] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.360138] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.360398] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.367374] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.392928] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.360191] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.360745] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.392953] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.367390] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.360641] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.392962] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.360762] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.367843] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.360829] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.392966] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.368025] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.360774] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.360845] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.368040] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.360875] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.360782] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.368069] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.360891] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.360802] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.368527] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.361362] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.360826] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.361546] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.361562] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.360898] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.360907] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.361591] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.360913] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.362039] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.360919] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.388830] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.381739] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.360924] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.381759] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.388849] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.453847] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.361022] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.381789] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.388879] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.361461] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.389338] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.361646] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.389521] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.361663] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.389536] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.361693] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.384328] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.389566] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.361709] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.384519] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.389584] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.384535] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.362168] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.390034] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.384564] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.362357] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.390221] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.384583] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.362374] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.362404] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.390251] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.385037] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.385224] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.362853] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.390700] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.363036] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.385254] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.390886] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.363051] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.390915] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.385728] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.363081] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.385914] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.391307] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.363548] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.385943] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.391488] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.363733] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.391774] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.363749] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.391977] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.363778] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392008] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.363797] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.392468] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.364256] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.364455] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.364493] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.364947] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.365133] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.365163] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.398406] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.365548] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.398603] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.365729] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.398854] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.366012] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.399039] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.474721] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.366200] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.476752] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.399069] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.366230] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.412703] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.399513] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.412744] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.366677] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.399698] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.366865] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.413226] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.399728] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.366895] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.413417] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.400186] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.413447] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.367346] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.400372] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.413905] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.400402] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.414168] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.400866] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.414185] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.401117] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.414214] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.401134] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.414679] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.401164] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.414862] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.401626] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.414877] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.401807] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.414907] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.401823] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.487692] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.415481] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.401852] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.415491] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.415497] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.411595] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.415541] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned client # [ 0.411636] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.415587] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.412127] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.415633] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.412416] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.415679] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.412434] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.412464] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.415725] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.412925] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.415771] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.413109] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.493585] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.415817] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.413125] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.495530] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.422519] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.415863] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.413154] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.422542] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.413723] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.415908] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.422548] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.413734] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.415977] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.422596] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.416022] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.413739] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.422645] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.416043] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.413785] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.422691] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.416064] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.413832] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.422739] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.416085] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.413879] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.416105] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.422786] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.413927] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.416126] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.422833] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.413973] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.416147] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.422879] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.414021] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.416171] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.422925] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.416192] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.414069] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.422972] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.416214] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.505552] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.414115] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.423073] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.416235] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.507429] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.414163] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.423127] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.509167] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.416257] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.414234] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.423149] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.416277] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.414280] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.423170] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.416298] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.414301] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.423192] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.416318] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.414323] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.423213] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.416339] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.416359] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned client # [ 0.414344] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.423234] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.416380] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.414365] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.423255] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.416406] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.414386] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.423283] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.416415] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.414407] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.423304] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.416419] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.414433] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.423326] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.414455] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.423348] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.414477] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.423370] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.414498] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.423391] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.423411] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.414520] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.519179] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.423432] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.414541] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.521037] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.414562] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.423453] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned client # [ 0.414583] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.423474] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned client # [ 0.414604] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.423496] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.414625] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.423521] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.461290] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.414646] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.423530] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.414672] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.423535] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.414681] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.424341] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.414686] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.532045] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.459560] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.542579] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.545172] msm_serial: driver initialized lb # [ 0.545339] SuperH (H)SCI(F) driver initialized lb # [ 0.546480] STM32 USART driver initialized worker1 # [ 0.486895] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.480274] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.493171] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.480311] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.495130] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.482407] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.490281] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.505084] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.493611] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.507133] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.495575] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.493400] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.497610] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.495316] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.578123] loop: module loaded lb # [ 0.578290] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.579074] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.507935] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.509664] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.505085] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.516700] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.508246] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.518981] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.510130] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.520751] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.522505] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.524294] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.520581] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.589986] megasas: 07.734.00.00-rc1 lb # [ 0.590637] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.523309] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.592845] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.525053] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.592868] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.519934] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.594614] Using buffer write method client # [ 0.521700] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.594675] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.538032] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.596446] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.596466] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.525035] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.527212] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.535466] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.616853] Using buffer write method worker1 # [ 0.553808] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.616887] Concatenating MTD devices: lb # [ 0.616891] (0): "0.flash" lb # [ 0.616895] (1): "0.flash" worker2 # [ 0.544549] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.616906] into device "0.flash" worker1 # [ 0.556328] msm_serial: driver initialized worker1 # [ 0.556488] SuperH (H)SCI(F) driver initialized client # [ 0.543516] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.547034] msm_serial: driver initialized worker2 # [ 0.547173] SuperH (H)SCI(F) driver initialized worker2 # [ 0.547227] STM32 USART driver initialized client # [ 0.546053] msm_serial: driver initialized client # [ 0.546184] SuperH (H)SCI(F) driver initialized client # [ 0.546237] STM32 USART driver initialized worker1 # [ 0.557666] STM32 USART driver initialized worker1 # [ 0.587352] loop: module loaded worker1 # [ 0.587533] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.588303] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.578320] loop: module loaded worker2 # [ 0.582684] loop: module loaded client # [ 0.578543] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.582876] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.579268] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.583613] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.593146] megasas: 07.734.00.00-rc1 worker1 # [ 0.593829] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.590690] megasas: 07.734.00.00-rc1 worker1 # [ 0.604574] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.591395] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.604601] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.606319] Using buffer write method worker2 # [ 0.593317] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.606404] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.593344] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.595019] Using buffer write method worker2 # [ 0.595085] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.592036] megasas: 07.734.00.00-rc1 client # [ 0.592815] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.597025] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.597047] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.594978] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.595000] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.596548] Using buffer write method client # [ 0.596620] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.617116] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.617139] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.598320] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.618398] Using buffer write method client # [ 0.598365] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.618423] Concatenating MTD devices: worker1 # [ 0.618427] (0): "0.flash" worker1 # [ 0.618431] (1): "0.flash" worker1 # [ 0.618439] into device "0.flash" client # [ 0.611940] Using buffer write method client # [ 0.611972] Concatenating MTD devices: client # [ 0.611976] (0): "0.flash" client # [ 0.611980] (1): "0.flash" worker2 # [ 0.617403] Using buffer write method client # [ 0.611987] into device "0.flash" worker2 # [ 0.617438] Concatenating MTD devices: worker2 # [ 0.617442] (0): "0.flash" worker2 # [ 0.617446] (1): "0.flash" worker2 # [ 0.617455] into device "0.flash" lb # [ 0.850867] Freeing initrd memory: 26948K lb # [ 0.856783] tun: Universal TUN/TAP device driver, 1.6 lb # [ 0.860408] thunder_xcv, ver 1.0 lb # [ 0.860455] thunder_bgx, ver 1.0 lb # [ 0.860477] nicpf, ver 1.0 lb # [ 0.861026] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.861033] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.861056] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.861064] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.861092] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.861097] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.861120] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 0.861126] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.861257] sky2: driver version 1.30 lb # [ 0.869911] usbcore: registered new interface driver usb-storage lb # [ 0.870003] usbcore: registered new interface driver usbserial_generic lb # [ 0.870016] usbserial: USB Serial support registered for generic lb # [ 0.870600] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.874741] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 0.874771] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 0.875014] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.877662] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.877691] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740) lb # [ 0.878028] i2c_dev: i2c /dev entries driver lb # [ 0.883341] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.883352] sdhci: Copyright(c) Pierre Ossman lb # [ 0.883615] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.883986] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.885419] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.885719] hub 1-0:1.0: USB hub found lb # [ 0.885750] hub 1-0:1.0: 6 ports detected lb # [ 0.890621] hid: raw HID events driver (C) Jiri Kosina lb # [ 0.890870] usbcore: registered new interface driver usbhid lb # [ 0.890876] usbhid: USB HID core driver lb # [ 0.893796] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.895295] drop_monitor: Initializing network drop monitor service lb # [ 0.895432] NET: Registered PF_INET6 protocol family lb # [ 0.898434] Segment Routing with IPv6 lb # [ 0.898451] In-situ OAM (IOAM) with IPv6 lb # [ 0.898478] NET: Registered PF_PACKET protocol family lb # [ 0.900139] 9pnet: Installing 9P2000 support lb # [ 0.900189] Key type dns_resolver registered lb # [ 0.906876] registered taskstats version 1 lb # [ 0.907012] Loading compiled-in X.509 certificates lb # [ 0.915820] Demotion targets for Node 0: null lb # [ 0.915934] Key type .fscrypt registered lb # [ 0.915941] Key type fscrypt-provisioning registered lb # [ 0.916034] ima: No TPM chip found, activating TPM-bypass! client # [ 0.839888] Freeing initrd memory: 26972K lb # [ 0.916053] ima: Allocated hash algorithm: sha1 lb # [ 0.916075] ima: No architecture policies found lb # [ 0.920070] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.845864] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.860953] Freeing initrd memory: 26976K worker2 # [ 0.852432] Freeing initrd memory: 26944K client # [ 0.849552] thunder_xcv, ver 1.0 client # [ 0.849595] thunder_bgx, ver 1.0 client # [ 0.849617] nicpf, ver 1.0 client # [ 0.850179] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.867180] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.850186] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.850210] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.858369] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.850218] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.850247] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.850253] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.850276] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.850282] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.870913] thunder_xcv, ver 1.0 worker2 # [ 0.862063] thunder_xcv, ver 1.0 client # [ 0.850417] sky2: driver version 1.30 worker1 # [ 0.870977] thunder_bgx, ver 1.0 worker1 # [ 0.871000] nicpf, ver 1.0 worker1 # [ 0.871567] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.859090] usbcore: registered new interface driver usb-storage worker1 # [ 0.871574] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.859178] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.871598] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.859192] usbserial: USB Serial support registered for generic worker1 # [ 0.871606] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.871635] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.871641] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.862388] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 0.937952] clk: Disabling unused clocks worker2 # [ 0.862575] thunder_bgx, ver 1.0 worker2 # [ 0.862606] nicpf, ver 1.0 worker1 # [ 0.871689] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 0.937980] PM: genpd: Disabling unused power domains worker2 # [ 0.863149] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.862418] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.871695] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.862711] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.863156] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.871833] sky2: driver version 1.30 worker2 # [ 0.863180] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.942089] Freeing unused kernel memory: 4736K worker2 # [ 0.863188] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.942298] Run /init as init process client # [ 0.865387] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.863221] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.863226] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.881212] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.866906] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.863250] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.881244] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.866933] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740) worker2 # [ 0.863256] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.881510] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.863392] sky2: driver version 1.30 client # [ 0.867226] i2c_dev: i2c /dev entries driver worker2 # [ 0.865029] usbcore: registered new interface driver usb-storage worker1 # [ 0.884009] usbcore: registered new interface driver usb-storage worker2 # [ 0.865151] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.884105] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.865166] usbserial: USB Serial support registered for generic worker1 # [ 0.884120] usbserial: USB Serial support registered for generic worker2 # [ 0.865750] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.866812] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.887068] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.866856] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.867045] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.888605] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.878832] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.872459] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.879155] hub 1-0:1.0: USB hub found client # [ 0.872469] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.888629] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740) worker2 # [ 0.879176] hub 1-0:1.0: 6 ports detected worker1 # [ 0.888936] i2c_dev: i2c /dev entries driver client # [ 0.872731] Synopsys Designware Multimedia Card Interface Driver client # [ 0.873094] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.881720] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.874591] hid: raw HID events driver (C) Jiri Kosina client # [ 0.874833] usbcore: registered new interface driver usbhid worker2 # [ 0.881750] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740) client # [ 0.874839] usbhid: USB HID core driver client # [ 0.879864] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.883809] i2c_dev: i2c /dev entries driver client # [ 0.880907] hub 1-0:1.0: USB hub found client # [ 0.881382] hub 1-0:1.0: 6 ports detected worker1 # [ 0.894389] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.959584] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.894401] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.894678] Synopsys Designware Multimedia Card Interface Driver client # [ 0.882751] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.895065] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.899311] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker1 # [ 0.900320] hub 1-0:1.0: USB hub found client # [ 0.885397] drop_monitor: Initializing network drop monitor service worker2 # [ 0.887467] sdhci: Secure Digital Host Controller Interface driver client # [ 0.885543] NET: Registered PF_INET6 protocol family worker1 # [ 0.900829] hub 1-0:1.0: 6 ports detected worker2 # [ 0.887479] sdhci: Copyright(c) Pierre Ossman client # [ 0.887391] Segment Routing with IPv6 worker2 # [ 0.887735] Synopsys Designware Multimedia Card Interface Driver client # [ 0.887420] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.888099] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.888415] NET: Registered PF_PACKET protocol family worker1 # [ 0.901798] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.889589] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.902063] usbcore: registered new interface driver usbhid worker2 # [ 0.889835] usbcore: registered new interface driver usbhid worker1 # [ 0.902069] usbhid: USB HID core driver worker2 # [ 0.889842] usbhid: USB HID core driver client # [ 0.889116] 9pnet: Installing 9P2000 support client # [ 0.889163] Key type dns_resolver registered worker1 # [ 0.905041] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.906603] drop_monitor: Initializing network drop monitor service worker2 # [ 0.895423] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.906781] NET: Registered PF_INET6 protocol family worker2 # [ 0.896943] drop_monitor: Initializing network drop monitor service worker2 # [ 0.897079] NET: Registered PF_INET6 protocol family worker1 # [ 0.909834] Segment Routing with IPv6 worker1 # [ 0.909854] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.900101] Segment Routing with IPv6 worker1 # [ 0.909884] NET: Registered PF_PACKET protocol family worker2 # [ 0.900120] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.900176] NET: Registered PF_PACKET protocol family worker1 # [ 0.911546] 9pnet: Installing 9P2000 support worker2 # [ 0.901817] 9pnet: Installing 9P2000 support worker1 # [ 0.911594] Key type dns_resolver registered worker2 # [ 0.901867] Key type dns_resolver registered client # [ 0.895777] registered taskstats version 1 client # [ 0.895947] Loading compiled-in X.509 certificates client # [ 0.904583] Demotion targets for Node 0: null client # [ 0.904684] Key type .fscrypt registered worker2 # [ 0.908559] registered taskstats version 1 client # [ 0.904690] Key type fscrypt-provisioning registered worker2 # [ 0.908721] Loading compiled-in X.509 certificates client # [ 0.904788] ima: No TPM chip found, activating TPM-bypass! client # [ 0.904807] ima: Allocated hash algorithm: sha1 client # [ 0.904829] ima: No architecture policies found client # [ 0.908840] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 0.918402] registered taskstats version 1 worker1 # [ 0.918546] Loading compiled-in X.509 certificates worker1 # [ 0.927059] Demotion targets for Node 0: null worker1 # [ 0.927168] Key type .fscrypt registered worker2 # [ 0.917254] Demotion targets for Node 0: null worker1 # [ 0.927174] Key type fscrypt-provisioning registered worker2 # [ 0.917365] Key type .fscrypt registered worker1 # [ 0.927271] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.917371] Key type fscrypt-provisioning registered worker1 # [ 0.927290] ima: Allocated hash algorithm: sha1 worker2 # [ 0.917467] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.927310] ima: No architecture policies found worker2 # [ 0.917486] ima: Allocated hash algorithm: sha1 worker2 # [ 0.917508] ima: No architecture policies found worker1 # [ 0.931820] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.921752] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.926398] clk: Disabling unused clocks client # [ 0.926425] PM: genpd: Disabling unused power domains client # [ 0.930594] Freeing unused kernel memory: 4736K client # [ 0.930803] Run /init as init process worker1 # [ 0.950124] clk: Disabling unused clocks worker2 # [ 0.940389] clk: Disabling unused clocks worker1 # [ 0.950155] PM: genpd: Disabling unused power domains worker2 # [ 0.940415] PM: genpd: Disabling unused power domains worker1 # [ 0.954365] Freeing unused kernel memory: 4736K worker1 # [ 0.954571] Run /init as init process worker2 # [ 0.944616] Freeing unused kernel memory: 4736K worker2 # [ 0.944822] Run /init as init process client # [ 0.945736] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 0.960254] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.970860] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.133476] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.126143] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.127513] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.148615] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.285980] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.294382] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.306655] systemd[1]: Detected virtualization qemu. lb # [ 1.308706] systemd[1]: Detected architecture arm64. lb # [ 1.310678] systemd[1]: Running in initrd. lb # [ 1.313261] systemd[1]: Initializing machine ID from random generator. lb # [ 1.316244] systemd[1]: Hostname set to . worker2 # [ 1.278647] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.280051] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.365651] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.301026] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.286068] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.298454] systemd[1]: Detected virtualization qemu. worker2 # [ 1.295123] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.298546] systemd[1]: Detected architecture arm64. worker1 # [ 1.307269] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.298571] systemd[1]: Running in initrd. worker2 # [ 1.307300] systemd[1]: Detected virtualization qemu. worker2 # [ 1.309400] systemd[1]: Detected architecture arm64. client # [ 1.305136] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.319861] systemd[1]: Detected virtualization qemu. worker2 # [ 1.311382] systemd[1]: Running in initrd. client # [ 1.307987] systemd[1]: Hostname set to . worker1 # [ 1.319950] systemd[1]: Detected architecture arm64. worker1 # [ 1.319973] systemd[1]: Running in initrd. worker2 # [ 1.313967] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.326626] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.316959] systemd[1]: Hostname set to . worker1 # [ 1.329655] systemd[1]: Hostname set to . worker2 # [ 1.362386] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.359691] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.380795] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.489430] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.482144] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.483474] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.504620] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.632869] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.645838] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.653686] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 1.619337] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.640154] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.630651] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.642531] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.646862] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.648267] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.647203] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.665308] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.671162] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.743171] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.751473] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.752502] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.753754] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.754662] systemd[1]: Reached target Path Units. lb # [ 1.755359] systemd[1]: Reached target Slice Units. lb # [ 1.756073] systemd[1]: Reached target Swaps. lb # [ 1.756716] systemd[1]: Reached target Timer Units. lb # [ 1.757610] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.758665] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.759647] systemd[1]: Listening on Journal Sockets. lb # [ 1.760491] systemd[1]: Listening on udev Control Socket. lb # [ 1.761360] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.762235] systemd[1]: Reached target Socket Units. lb # [ 1.764380] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.765360] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.773572] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.781779] systemd[1]: Starting Journal Service... lb # [ 1.797236] systemd[1]: Starting Load Kernel Modules... lb # [ 1.801682] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.734539] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.740792] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.741625] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.755969] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.742681] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.819343] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.743921] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.823478] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.743954] systemd[1]: Reached target Path Units. client # [ 1.743971] systemd[1]: Reached target Slice Units. lb # [ 1.824933] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.743988] systemd[1]: Reached target Swaps. client # [ 1.744004] systemd[1]: Reached target Timer Units. worker2 # [ 1.752816] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.744173] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.744308] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.744442] systemd[1]: Listening on Journal Sockets. client # [ 1.744565] systemd[1]: Listening on udev Control Socket. client # [ 1.744667] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.767020] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.744687] systemd[1]: Reached target Socket Units. worker2 # [ 1.754352] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.768216] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.754403] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.754376] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.754438] systemd[1]: Reached target Path Units. client # [ 1.755395] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.754460] systemd[1]: Reached target Slice Units. worker2 # [ 1.754482] systemd[1]: Reached target Swaps. worker1 # [ 1.769686] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.754504] systemd[1]: Reached target Timer Units. worker1 # [ 1.769723] systemd[1]: Reached target Path Units. worker1 # [ 1.769743] systemd[1]: Reached target Slice Units. worker2 # [ 1.754712] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.769762] systemd[1]: Reached target Swaps. worker2 # [ 1.754881] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.769783] systemd[1]: Reached target Timer Units. worker2 # [ 1.755047] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.769972] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.755195] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.770129] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.755328] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.770273] systemd[1]: Listening on Journal Sockets. client # [ 1.763589] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.755353] systemd[1]: Reached target Socket Units. worker1 # [ 1.770406] systemd[1]: Listening on udev Control Socket. lb # [ 1.839335] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.770518] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.769583] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.839854] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.770540] systemd[1]: Reached target Socket Units. worker1 # [ 1.781811] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.770968] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.783009] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.771682] systemd[1]: Starting Journal Service... worker2 # [ 1.778986] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.792708] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.857482] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.800833] systemd[1]: Starting Journal Service... worker2 # [ 1.790404] systemd[1]: Starting Journal Service... lb # [ 1.871264] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.871491] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.871499] [drm] features: -context_init lb # [ 1.872174] [drm] number of scanouts: 1 lb # [ 1.872191] [drm] number of cap sets: 0 client # [ 1.799587] systemd[1]: Starting Load Kernel Modules... client # [ 1.800415] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.803785] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.882589] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.824771] systemd[1]: Starting Load Kernel Modules... client # [ 1.813419] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.814460] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.828792] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.890246] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.890263] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.822369] systemd[1]: Starting Load Kernel Modules... lb # [ 1.895714] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.822502] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.898318] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.820008] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.825955] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.844763] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.837705] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.839997] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.855371] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 1.850225] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.850914] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.855678] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.868737] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.921678] Console: switching to colour frame buffer device 160x50 worker1 # [ 1.869404] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.928109] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 1.884926] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.950604] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.950799] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.950830] systemd[1]: Reached target Local File Systems. client # [ 1.877664] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.955566] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.887550] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.965601] systemd[1]: Finished Load Kernel Modules. client # [ 1.890021] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.969808] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.910553] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.899603] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.904112] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.906423] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.910452] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.925162] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.927570] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.916979] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.917212] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.917221] [drm] features: -context_init client # [ 1.917913] [drm] number of scanouts: 1 client # [ 1.917930] [drm] number of cap sets: 0 worker1 # [ 1.936696] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.934230] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.934248] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 1.950384] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.950616] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.937800] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.950625] [drm] features: -context_init worker2 # [ 1.930866] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.951331] [drm] number of scanouts: 1 client # [ 1.938903] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.937081] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.951349] [drm] number of cap sets: 0 client # [ 1.939811] systemd[1]: Reached target Local File Systems. worker2 # [ 1.937311] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.937322] [drm] features: -context_init worker2 # [ 1.938022] [drm] number of scanouts: 1 worker2 # [ 1.938041] [drm] number of cap sets: 0 lb # [ 2.017986] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 1.967671] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.967975] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.951806] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.968001] systemd[1]: Reached target Local File Systems. worker2 # [ 1.958677] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.958997] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.971920] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.959024] systemd[1]: Reached target Local File Systems. lb # [ 2.037111] systemd[1]: Started Journal Service. worker2 # [ 1.962807] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.963756] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.970528] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.970544] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 1.984763] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.984779] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 2.040294] systemd-modules-load[73]: Inserted module 'dm_mod' lb # [ 2.041548] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 2.042601] systemd-modules-load[73]: Module 'virtio_console' is built in lb # [ 2.043648] systemd-modules-load[73]: Inserted module 'virtio_gpu' client # [ 1.981703] systemd[1]: Started Journal Service. lb # [ 2.056117] systemd-modules-load[73]: Module 'virtio_rng' is built in worker2 # [ 1.990307] Console: switching to colour frame buffer device 160x50 worker1 # [ 2.000926] Console: switching to colour frame buffer device 160x50 lb # [ 2.058812] systemd[1]: Starting Create System Files and Directories... worker1 # [ 2.007263] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 2.000029] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.987744] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 1.989095] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.068859] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 1.990485] systemd-modules-load[74]: Module 'virtio_console' is built in client # [ 1.991513] systemd[1]: Starting Create System Files and Directories... lb # [ 2.071691] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.018640] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 2.023332] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.033870] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.012533] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.008479] systemd-modules-load[74]: Inserted module 'virtio_gpu' client # [ 2.009543] systemd-modules-load[74]: Module 'virtio_rng' is built in client # [ 2.010538] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.030520] systemd-modules-load[74]: Inserted module 'dm_mod' lb # [ 2.092214] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.044790] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.034574] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.034796] systemd[1]: Started Journal Service. worker1 # [ 2.046978] systemd[1]: Started Journal Service. worker2 # [ 2.023165] systemd-modules-load[73]: Module 'virtio_balloon' is built in client # [ 2.020617] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.044241] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker2 # [ 2.032356] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.033533] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker1 # [ 2.045462] systemd-modules-load[74]: Module 'virtio_console' is built in worker2 # [ 2.034558] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 2.046779] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 2.047772] systemd-modules-load[74]: Module 'virtio_rng' is built in client # [ 2.033460] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.052123] systemd[1]: Starting Create System Files and Directories... client # [ 2.049019] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.054490] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 2.052478] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker1 # [ 2.072149] systemd[1]: Starting Create System Files and Directories... worker1 # [ 2.082627] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker1 # [ 2.089465] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.152204] systemd[1]: Starting Virtual Console Setup... client # [ 2.078242] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.089191] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.100316] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.113171] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.102213] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.123494] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.200511] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.203870] systemd[1]: Finished Virtual Console Setup. client # [ 2.131669] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.164156] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.182039] systemd[1]: Starting Virtual Console Setup... client # [ 2.184498] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.187985] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.208422] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.211792] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.228457] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.231764] systemd[1]: Finished Virtual Console Setup. lb # [ 2.801138] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.802085] systemd[1]: Reached target System Initialization. lb # [ 2.802893] systemd[1]: Reached target Basic System. client # [ 2.790654] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.791930] systemd[1]: Reached target System Initialization. client # [ 2.796459] systemd[1]: Reached target Basic System. worker2 # [ 2.809901] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.810886] systemd[1]: Reached target System Initialization. worker2 # [ 2.816296] systemd[1]: Reached target Basic System. worker1 # [ 2.834121] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.836146] systemd[1]: Reached target System Initialization. worker1 # [ 2.840101] systemd[1]: Reached target Basic System. lb # [ 2.939386] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.977563] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.982080] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.939725] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.947953] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.959537] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.040122] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.044411] systemd[1]: Reached target Initrd Root Device. lb # [ 3.047827] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 2.990237] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.007811] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.011917] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.000775] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.004150] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.017960] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.103210] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.108244] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.118877] systemd[1]: Mounting /sysroot... worker2 # [ 3.068397] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.065212] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.067010] systemd[1]: Reached target Initrd Root Device. client # [ 3.069987] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.075762] systemd[1]: Reached target Initrd Root Device. lb # [ 3.163899] EXT4-fs (vda): mounted filesystem 4e796412-306a-4bfb-ae60-e56a025ef008 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.082149] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.157017] systemd[1]: Mounted /sysroot. lb # [ 3.159002] systemd[1]: Reached target Initrd Root File System. lb # [ 3.162508] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.106861] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.108638] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.116099] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.188570] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.195988] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.197673] systemd[1]: Reloading... client # [ 3.118382] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.130933] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.126718] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.132988] systemd[1]: Mounting /sysroot... worker2 # [ 3.139520] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.152687] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.144919] systemd[1]: Mounting /sysroot... worker1 # [ 3.158084] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.167989] systemd[1]: Mounting /sysroot... client # [ 3.180376] EXT4-fs (vda): mounted filesystem ae37c38b-e469-4e07-87a2-8424a7829175 r/w with ordered data mode. Quota mode: none. client # [ 3.170059] systemd[1]: Mounted /sysroot. client # [ 3.172714] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.191700] EXT4-fs (vda): mounted filesystem db3a8e93-7bbf-4dc9-b691-45970539d6f1 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.184182] systemd[1]: Mounted /sysroot. client # [ 3.179883] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.185989] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.211216] EXT4-fs (vda): mounted filesystem 933f64f8-b512-4fd9-b502-2906bfb81b78 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.192129] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.205609] systemd[1]: Mounted /sysroot. worker1 # [ 3.207822] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.213076] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.199539] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.204811] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.209530] systemd[1]: Reloading... worker2 # [ 3.215671] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.221880] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.227054] systemd[1]: Reloading... worker1 # [ 3.239724] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.245571] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.250392] systemd[1]: Reloading... lb # [ 3.407480] systemd[1]: Reloading finished in 208 ms. lb # [ 3.435827] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.440335] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.447185] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.449608] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.451587] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.416110] systemd[1]: Reloading finished in 205 ms. worker2 # [ 3.429525] systemd[1]: Reloading finished in 204 ms. worker1 # [ 3.454631] systemd[1]: Reloading finished in 206 ms. client # [ 3.447567] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.452200] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.460100] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.464231] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.458961] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.461875] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.479470] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.463368] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.471365] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.484165] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.473970] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.476146] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.490574] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.493534] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.495181] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.812366] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.826900] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.833644] systemd[1]: Mounting /sysroot/run... lb # [ 3.852707] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.793668] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.877555] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.810745] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.815206] systemd[1]: Mounting /sysroot/run... worker2 # [ 3.836971] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.833865] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.854003] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.934019] fuse: init (API version 7.45) lb # [ 3.940285] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 3.856098] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.941042] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.859375] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.872208] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.876315] systemd[1]: Mounting /sysroot/run... lb # [ 3.939344] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.955232] virtiofs virtio7: discovered new tag: shared lb # [ 3.955954] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 3.865914] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.874388] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.948551] systemd[1]: Mounted /sysroot/run. lb # [ 3.966845] virtiofs virtio8: discovered new tag: xchg lb # [ 3.967559] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.897517] fuse: init (API version 7.45) lb # [ 3.963362] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.895095] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.908576] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.975691] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.911678] virtiofs virtio6: discovered new tag: nix-store client # [ 3.912480] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 3.982102] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 3.903112] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.921595] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.983652] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 3.929157] virtiofs virtio7: discovered new tag: shared client # [ 3.929898] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 3.999498] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.001691] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.936900] virtiofs virtio8: discovered new tag: xchg client # [ 3.937618] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.005973] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 3.936377] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.950531] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.934554] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.971807] fuse: init (API version 7.45) worker2 # [ 3.956447] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.976199] fuse: init (API version 7.45) worker1 # [ 3.988949] virtiofs virtio6: discovered new tag: nix-store client # [ 3.961333] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.989742] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.967848] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.985363] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.987183] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 3.987960] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.971491] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.053247] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.974509] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.054179] systemd[1]: Reached target Initrd File Systems. worker2 # [ 3.981034] systemd[1]: Mounted /sysroot/run. client # [ 3.975790] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.056170] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.006454] virtiofs virtio7: discovered new tag: shared worker1 # [ 4.007224] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 3.979431] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.003752] virtiofs virtio7: discovered new tag: shared worker1 # [ 4.013699] virtiofs virtio8: discovered new tag: xchg worker2 # [ 4.004509] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 4.014439] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.067000] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.006690] systemd[1]: Mounted /sysroot/run. worker2 # [ 4.011172] virtiofs virtio8: discovered new tag: xchg worker2 # [ 4.011894] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.014041] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 4.027997] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.018927] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 4.033353] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.095136] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.022446] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.025807] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 4.040944] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.029255] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.046929] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.109254] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.049127] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.112232] systemd[1]: Reached target Initrd Default Target. lb # [ 4.117826] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.141757] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.145343] systemd[1]: Stopped target Basic System. lb # [ 4.146329] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.147325] systemd[1]: Stopped target Path Units. lb # [ 4.150885] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.154218] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.156641] systemd[1]: Stopped target Slice Units. lb # [ 4.160350] systemd[1]: Stopped target Socket Units. lb # [ 4.162162] systemd[1]: Stopped target System Initialization. lb # [ 4.164360] systemd[1]: Stopped target Swaps. lb # [ 4.166758] systemd[1]: Stopped target Timer Units. lb # [ 4.170520] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.172175] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.177419] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.178524] systemd[1]: Stopped Find NixOS closure. lb # [ 4.181056] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.184641] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.186530] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.188962] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.193586] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.194453] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 4.198549] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.199722] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.205532] systemd[1]: Stopped Create System Files and Directories. lb # [ 4.206467] systemd[1]: Stopped target Local File Systems. lb # [ 4.209812] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.213439] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.215179] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.216983] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.219516] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.221678] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.224378] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 4.225343] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 4.228143] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.229247] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.230073] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.230975] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.231944] systemd[1]: systemd-udevd.service: Consumed 1.112s CPU time over 2.274s wall clock time, 18.1M memory peak. lb # [ 4.233527] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 4.234507] systemd[1]: Closed udev Control Socket. lb # [ 4.235201] systemd[1]: Starting Cleanup udev Database... lb # [ 4.235949] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 4.244287] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.245196] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.246536] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.252138] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.253109] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 4.273336] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 4.275645] systemd[1]: Finished Cleanup udev Database. lb # [ 4.277406] systemd[1]: Reached target Switch Root. lb # [ 4.280289] systemd[1]: Starting NixOS Activation... lb # [ 4.356396] initrd-nixos-activation-start[168]: booting system configuration /nix/store/3in2qynk4v72bmzzvgahb746q83gs855-nixos-system-lb-test lb # [ 4.389571] initrd-nixos-activation-start[168]: running activation script... client # [ 4.334626] (udev-worker)[92]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.341295] (udev-worker)[92]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.356623] (udev-worker)[92]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.372726] (udev-worker)[97]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.364202] (udev-worker)[92]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.378295] (udev-worker)[97]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.373063] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.374813] systemd[1]: Stopped Virtual Console Setup. client # [ 4.376724] systemd[1]: Stopping Virtual Console Setup... client # [ 4.379886] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.393152] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.397365] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.399206] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.414938] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.404379] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.416483] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.418364] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.419868] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.410203] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.411683] systemd[1]: Stopped Virtual Console Setup. client # [ 4.409025] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.417378] systemd[1]: Starting Virtual Console Setup... client # [ 4.412201] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.453521] systemd-vconsole-setup[151]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.456815] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.444575] systemd-vconsole-setup[155]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.447454] systemd[1]: Finished Virtual Console Setup. lb # [ 4.616531] initrd-nixos-activation-start[191]: setting up /etc... lb # [ 4.730254] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 4.731815] systemd[1]: Finished NixOS Activation. lb # [ 4.734254] systemd[1]: Starting Switch Root... lb # [ 4.754644] systemd[1]: Switching root. client # [ 4.796394] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.839436] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.855107] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.935254] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 4.865816] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.868984] systemd[1]: Reached target Initrd File Systems. client # [ 4.874840] systemd[1]: Starting Find NixOS closure... client # [ 4.884818] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 4.904927] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.917566] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.907969] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.920793] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.912570] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.925012] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.924327] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.936344] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.929249] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.936384] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.944856] systemd[1]: Finished Find NixOS closure. client # [ 4.947745] systemd[1]: Reached target Initrd Default Target. client # [ 4.949690] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 4.961060] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.976146] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.967655] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.980381] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 4.975944] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.978878] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.980728] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.975972] systemd[1]: Stopped target Initrd Default Target. client # [ 4.977918] systemd[1]: Stopped target Basic System. worker1 # [ 4.996205] systemd[1]: Finished Find NixOS closure. client # [ 4.979465] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.999125] systemd[1]: Reached target Initrd Default Target. client # [ 4.984207] systemd[1]: Stopped target Path Units. worker1 # [ 5.001316] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.985228] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.986739] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 4.989602] systemd[1]: Stopped target Slice Units. client # [ 4.992171] systemd[1]: Stopped target Socket Units. client # [ 4.994132] systemd[1]: Stopped target System Initialization. client # [ 4.999378] systemd[1]: Stopped target Swaps. client # [ 5.000393] systemd[1]: Stopped target Timer Units. client # [ 5.001333] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.002389] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.003429] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 5.009508] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 5.011309] systemd[1]: Stopped target Basic System. worker2 # [ 5.012643] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.016515] systemd[1]: Stopped target Path Units. worker2 # [ 5.017527] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 5.020112] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.016701] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.033795] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 5.022589] systemd[1]: Stopped target Slice Units. worker1 # [ 5.035920] systemd[1]: Stopped target Basic System. worker2 # [ 5.024444] systemd[1]: Stopped target Socket Units. client # [ 5.021433] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.027973] systemd[1]: Stopped target System Initialization. worker1 # [ 5.040711] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.041885] systemd[1]: Stopped target Path Units. client # [ 5.024649] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 5.042916] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 5.032229] systemd[1]: Stopped target Swaps. worker2 # [ 5.033973] systemd[1]: Stopped target Timer Units. client # [ 5.028614] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.036494] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.048302] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.034435] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 5.052418] systemd[1]: Stopped target Slice Units. client # [ 5.035504] systemd[1]: Stopped Load Kernel Modules. client # [ 5.037588] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.056184] systemd[1]: Stopped target Socket Units. client # [ 5.039218] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.057189] systemd[1]: Stopped target System Initialization. worker2 # [ 5.045719] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.040578] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.058703] systemd[1]: Stopped target Swaps. worker2 # [ 5.046679] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.041598] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.047825] systemd[1]: Stopped Find NixOS closure. client # [ 5.042446] systemd[1]: Stopped target Local File Systems. client # [ 5.043215] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 5.061715] systemd[1]: Stopped target Timer Units. worker2 # [ 5.052735] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.064695] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.048566] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.054469] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.055550] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.052267] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.070207] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.053111] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.072988] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.056405] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.057555] systemd[1]: Stopped Virtual Console Setup. client # [ 5.058293] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.059202] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.077433] systemd[1]: Stopped Find NixOS closure. client # [ 5.060254] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 5.061243] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.062048] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.067531] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.062951] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.069887] systemd[1]: Stopped Load Kernel Modules. client # [ 5.063922] systemd[1]: systemd-udevd.service: Consumed 1.414s CPU time over 3.110s wall clock time, 25.4M memory peak. worker2 # [ 5.072456] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.073987] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.069503] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.087225] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.077238] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.072159] systemd[1]: Closed udev Control Socket. worker1 # [ 5.089470] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 5.090542] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 5.091356] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.080378] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.081341] systemd[1]: Stopped target Local File Systems. client # [ 5.076284] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.082668] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.077286] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.078353] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.096365] systemd[1]: Stopped Load Kernel Modules. client # [ 5.079219] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.085617] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.086756] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.098456] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.087563] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.088768] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.089792] systemd[1]: Stopped Virtual Console Setup. client # [ 5.084245] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.090540] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.085250] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.091469] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.086190] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.103572] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.106515] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.108137] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.100301] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.112525] systemd[1]: Stopped target Local File Systems. worker2 # [ 5.101263] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.113378] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 5.114328] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.102542] systemd[1]: systemd-udevd.service: Consumed 1.416s CPU time over 3.118s wall clock time, 22.4M memory peak. worker1 # [ 5.116295] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.117547] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.118693] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.119724] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.108190] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.120578] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 5.109207] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.121524] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.110021] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.122445] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.111013] systemd[1]: Closed udev Control Socket. worker1 # [ 5.123437] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.111716] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.128230] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.116465] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.117547] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.112839] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.118419] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.120172] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.132288] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.115134] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.133480] systemd[1]: systemd-udevd.service: Consumed 1.407s CPU time over 3.143s wall clock time, 22.3M memory peak. worker1 # [ 5.136350] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.124418] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.125412] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.119942] systemd[1]: Reached target Switch Root. client # [ 5.120844] systemd[1]: Starting NixOS Activation... worker1 # [ 5.140139] systemd[1]: Closed udev Control Socket. worker1 # [ 5.140867] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.141630] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 5.144126] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.148326] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.149478] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.150459] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.152254] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.145208] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.148556] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.149359] systemd[1]: Reached target Switch Root. worker2 # [ 5.152501] systemd[1]: Starting NixOS Activation... worker1 # [ 5.174958] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 5.177828] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.180819] systemd[1]: Reached target Switch Root. worker1 # [ 5.184186] systemd[1]: Starting NixOS Activation... client # [ 5.198935] initrd-nixos-activation-start[176]: booting system configuration /nix/store/1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test worker2 # [ 5.230483] initrd-nixos-activation-start[178]: booting system configuration /nix/store/yhybdyymi2isy70630z4y9gwglld4pzg-nixos-system-worker2-test client # [ 5.230765] initrd-nixos-activation-start[176]: running activation script... worker1 # [ 5.263341] initrd-nixos-activation-start[174]: booting system configuration /nix/store/waj0blgjl9bpqbg9b6xiyknfyf8i56g7-nixos-system-worker1-test worker2 # [ 5.262559] initrd-nixos-activation-start[178]: running activation script... worker1 # [ 5.295968] initrd-nixos-activation-start[174]: running activation script... lb # [ 5.454057] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 5.466885] systemd[1]: Detected virtualization qemu. lb # [ 5.470029] systemd[1]: Detected architecture arm64. lb # [ 5.474093] systemd[1]: Detected first boot. lb # [ 5.479732] systemd[1]: Initializing machine ID from random generator. client # [ 5.468509] initrd-nixos-activation-start[199]: setting up /etc... worker2 # [ 5.502431] initrd-nixos-activation-start[201]: setting up /etc... worker1 # [ 5.565821] initrd-nixos-activation-start[197]: setting up /etc... client # [ 5.584586] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.587281] systemd[1]: Finished NixOS Activation. client # [ 5.592207] systemd[1]: Starting Switch Root... client # [ 5.610876] systemd[1]: Switching root. worker2 # [ 5.619151] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.621922] systemd[1]: Finished NixOS Activation. worker2 # [ 5.624154] systemd[1]: Starting Switch Root... worker2 # [ 5.642774] systemd[1]: Switching root. worker1 # [ 5.683927] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.686898] systemd[1]: Finished NixOS Activation. worker1 # [ 5.692262] systemd[1]: Starting Switch Root... worker1 # [ 5.711285] systemd[1]: Switching root. lb # [ 5.803022] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 5.789293] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.836018] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.894009] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 6.009190] systemd[1]: Applying preset policy. lb # [ 6.265640] systemd[1]: Populated /etc with preset unit settings. client # [ 6.323269] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.335645] systemd[1]: Detected virtualization qemu. client # [ 6.338829] systemd[1]: Detected architecture arm64. client # [ 6.342684] systemd[1]: Detected first boot. client # [ 6.349046] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.367170] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.379949] systemd[1]: Detected virtualization qemu. worker2 # [ 6.383157] systemd[1]: Detected architecture arm64. worker2 # [ 6.387128] systemd[1]: Detected first boot. worker2 # [ 6.393387] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.426326] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.439276] systemd[1]: Detected virtualization qemu. worker1 # [ 6.442575] systemd[1]: Detected architecture arm64. worker1 # [ 6.446724] systemd[1]: Detected first boot. worker1 # [ 6.453948] systemd[1]: Initializing machine ID from random generator. lb # [ 6.520589] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.522420] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.527246] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.531593] systemd[1]: Created slice Slice /system/getty. lb # [ 6.533850] systemd[1]: Created slice User and Session Slice. lb # [ 6.535089] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 6.537913] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 6.540121] systemd[1]: Expecting device /dev/hvc0... lb # [ 6.542139] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 6.544052] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 6.546186] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.547226] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 6.548830] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 6.550638] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 6.553140] systemd[1]: Reached target Path Units. lb # [ 6.554136] systemd[1]: Reached target Remote File Systems. lb # [ 6.555708] systemd[1]: Reached target Slice Units. lb # [ 6.557229] systemd[1]: Reached target Swaps. lb # [ 6.561011] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 6.564438] systemd[1]: Listening on Process Core Dump Socket. lb # [ 6.566700] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 6.569252] systemd[1]: Listening on Factory Reset Management. lb # [ 6.570659] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.575052] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.577244] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.581045] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.582840] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 6.584987] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.587000] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 6.593785] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 6.595164] systemd[1]: Listening on udev Control Socket. lb # [ 6.596790] systemd[1]: Listening on udev Varlink Socket. lb # [ 6.600705] systemd[1]: Mounting Huge Pages File System... lb # [ 6.604347] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 6.614144] systemd[1]: Mounting Kernel Debug File System... lb # [ 6.627835] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.633833] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.637526] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 6.657693] systemd[1]: Mounting Kernel Configuration File System... lb # [ 6.659938] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 6.665538] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 6.668388] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.697967] systemd[1]: Mounting FUSE Control File System... lb # [ 6.701843] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 6.738699] systemd[1]: Starting Journal Service... lb # [ 6.755421] systemd[1]: Starting Load Kernel Modules... client # [ 6.695507] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.778961] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 6.790466] systemd-journald[262]: Collecting audit messages is enabled. lb # [ 6.802269] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 6.804121] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.793568] systemd[1]: Queued start job for default target Multi-User System. lb # [ 6.795315] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 6.745333] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.837084] systemd[1]: Starting Coldplug All udev Devices... lb # [ 6.841247] systemd[1]: Started Journal Service. lb # [ 6.835913] systemd-modules-load[264]: Module 'atkbd' is built in lb # [ 6.837849] systemd-modules-load[264]: Module 'loop' is built in lb # [ 6.848344] systemd[1]: Listening on Journal Log Access Socket. lb # [ 6.849287] systemd[1]: Mounted Huge Pages File System. lb # [ 6.850024] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 6.850852] systemd[1]: Mounted Kernel Debug File System. lb # [ 6.851606] systemd[1]: Mounted Kernel Trace File System. lb # [ 6.860655] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 6.861570] systemd[1]: Mounted Kernel Configuration File System. lb # [ 6.870771] systemd[1]: Finished Load Kernel Modules. worker1 # [ 6.819191] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.886069] systemd[1]: Starting Firewall... lb # [ 6.889394] systemd[1]: Starting Apply Kernel Variables... lb # [ 6.896616] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 6.942425] EXT4-fs (vda): re-mounted 4e796412-306a-4bfb-ae60-e56a025ef008. lb # [ 6.949973] systemd-oomd[265]: No swap; memory pressure usage will be degraded lb # [ 6.969397] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 6.972695] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 6.983374] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 6.987801] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 6.992445] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.003923] systemd[1]: Starting Load/Save OS Random Seed... client # [ 6.942879] systemd[1]: Applying preset policy. lb # [ 7.010081] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.011313] systemd[1]: Mounted FUSE Control File System. worker2 # [ 7.002892] systemd[1]: Applying preset policy. worker1 # [ 7.032118] systemd[1]: Applying preset policy. lb # [ 7.095868] systemd-journald[262]: Received client request to flush runtime journal. lb # [ 7.144739] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.145638] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.146414] systemd[1]: Reached target First Boot Complete. lb # [ 7.153577] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 7.198429] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 7.204614] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 7.251987] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.277167] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.308270] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.452488] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 7.453886] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 7.462832] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 7.519917] systemd[1]: Mounting /run/wrappers... client # [ 7.523036] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.527887] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.524531] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.529336] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.528433] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.534021] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.531192] systemd[1]: Created slice Slice /system/getty. client # [ 7.533901] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.538983] systemd[1]: Created slice Slice /system/getty. client # [ 7.535648] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.600873] systemd[1]: Mounted /run/wrappers. worker2 # [ 7.541341] systemd[1]: Created slice User and Session Slice. lb # [ 7.601697] systemd[1]: Reached target Local File Systems. client # [ 7.537321] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.539056] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.542757] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.540788] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.545329] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.543074] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.558227] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.547625] systemd[1]: Expecting device /dev/hvc0... client # [ 7.544311] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.549438] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.559622] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.546632] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.551423] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.547809] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.552607] systemd[1]: Stopped target initrd-fs.target. client # [ 7.550234] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.554956] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.564473] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.556113] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.551504] systemd[1]: Reached target Path Units. client # [ 7.553752] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.557744] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.569172] systemd[1]: Created slice Slice /system/getty. lb # [ 7.618591] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 7.554866] systemd[1]: Reached target Slice Units. worker2 # [ 7.559584] systemd[1]: Reached target Path Units. worker1 # [ 7.571384] systemd[1]: Created slice User and Session Slice. client # [ 7.557315] systemd[1]: Reached target Swaps. worker2 # [ 7.561673] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.562874] systemd[1]: Reached target Slice Units. worker1 # [ 7.572767] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.560067] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.564294] systemd[1]: Reached target Swaps. lb # [ 7.624411] systemd[1]: Starting register-nix-paths.service... worker1 # [ 7.575477] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.567297] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.563626] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.577948] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.579896] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.570446] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.566245] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.632172] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 7.581951] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.572888] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.568906] systemd[1]: Listening on Factory Reset Management. lb # [ 7.633091] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.584071] systemd[1]: Stopped target initrd-fs.target. client # [ 7.570154] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.575414] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.586126] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.576706] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.588146] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.574884] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.590269] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.581141] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.592398] systemd[1]: Reached target Path Units. client # [ 7.577642] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.583367] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.594281] systemd[1]: Reached target Remote File Systems. client # [ 7.581951] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.596246] systemd[1]: Reached target Slice Units. worker2 # [ 7.587253] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.583808] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.598167] systemd[1]: Reached target Swaps. worker2 # [ 7.588870] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.586553] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.601582] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.590531] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.652139] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 7.593625] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.604797] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.589253] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.607130] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.609930] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.599962] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 7.596154] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.611214] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.601365] systemd[1]: Listening on udev Control Socket. client # [ 7.597701] systemd[1]: Listening on udev Control Socket. lb # [ 7.661809] systemd[1]: Starting Create System Files and Directories... client # [ 7.599236] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.603034] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.615751] systemd[1]: Starting Journal Log Access Socket... client # [ 7.603494] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.608024] systemd[1]: Mounting Huge Pages File System... worker1 # [ 7.617994] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.621928] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.623838] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.626044] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.611904] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.616562] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.628035] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.634479] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.635921] systemd[1]: Listening on udev Control Socket. client # [ 7.623229] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.637710] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.628858] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.642497] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.639107] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.649295] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.636764] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.660133] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.654445] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.652453] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.654862] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.654721] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.676701] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.673785] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.674335] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.674740] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.675129] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.690120] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.692481] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.686142] systemd[1]: Mounting Kernel Configuration File System... client # [ 7.687852] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.697288] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.702277] systemd[1]: Mounting FUSE Control File System... client # [ 7.699296] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.714808] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.715218] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.715501] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.715812] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.706770] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.716601] systemd[1]: Mounting FUSE Control File System... client # [ 7.719689] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.743154] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.748896] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 7.741236] systemd[1]: Starting Journal Service... client # [ 7.746803] systemd[1]: Starting Journal Service... worker1 # [ 7.767315] systemd[1]: Starting Journal Service... lb # [ 7.817608] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 7.762144] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.768012] systemd[1]: Starting Load Kernel Modules... lb # [ 7.834616] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 7.787160] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.778896] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.845567] systemd-udevd[302]: Using default interface naming scheme 'v261'. client # [ 7.781265] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.795196] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.807050] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.797027] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.816787] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.818145] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.811965] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.808536] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.811890] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.829333] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.819967] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.824486] systemd[1]: Mounted Huge Pages File System. worker1 # [ 7.837971] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.827808] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.840441] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.831562] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.842780] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.845201] systemd[1]: Mounted Kernel Debug File System. client # [ 7.831385] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.834404] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.834954] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.835458] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.848053] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.853371] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.855722] systemd[1]: Mounted FUSE Control File System. client # [ 7.843345] systemd[1]: Listening on Journal Log Access Socket. client # [ 7.850508] systemd[1]: Mounted Huge Pages File System. client # [ 7.855253] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.858379] systemd[1]: Mounted Kernel Debug File System. client # [ 7.859133] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.880881] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.889588] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.882371] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 7.886785] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 7.949946] systemd[1]: Finished Create System Files and Directories. client # [ 7.887975] systemd[1]: Mounted Kernel Configuration File System. client # [ 7.890465] systemd[1]: Mounted FUSE Control File System. worker2 # [ 7.895384] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.894575] systemd-journald[269]: Collecting audit messages is enabled. lb # [ 7.964654] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 7.974698] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 7.911629] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 7.946343] systemd-journald[267]: Collecting audit messages is enabled. client # [ 7.918226] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.937762] EXT4-fs (vda): re-mounted db3a8e93-7bbf-4dc9-b691-45970539d6f1. client # [ 7.936566] systemd[1]: Started Journal Service. client # [ 7.924880] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 7.926306] systemd-modules-load[270]: Module 'atkbd' is built in client # [ 7.927337] systemd-modules-load[270]: Module 'loop' is built in worker2 # [ 7.945575] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.959415] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.948291] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.948719] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 7.956764] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.939497] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.957140] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.969574] systemd[1]: Starting Firewall... worker2 # [ 7.962025] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.967954] systemd[1]: Starting Firewall... worker1 # [ 7.978318] systemd[1]: Starting Apply Kernel Variables... client # [ 7.952010] systemd[1]: Starting Apply Kernel Variables... client # [ 7.969171] EXT4-fs (vda): re-mounted ae37c38b-e469-4e07-87a2-8424a7829175. worker1 # [ 7.985338] EXT4-fs (vda): re-mounted 933f64f8-b512-4fd9-b502-2906bfb81b78. worker2 # [ 7.982841] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.985929] systemd-journald[271]: Collecting audit messages is enabled. client # [ 7.967898] systemd-oomd[271]: No swap; memory pressure usage will be degraded worker1 # [ 8.002478] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 8.003392] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.003826] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 8.003556] systemd[1]: Queued start job for default target Multi-User System. client # [ 7.990901] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 7.991936] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.022689] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 8.026178] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.029003] systemd[1]: Started Journal Service. client # [ 8.000350] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.021494] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 8.007092] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 8.012290] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 8.030552] systemd-modules-load[268]: Module 'atkbd' is built in worker2 # [ 8.031687] systemd[1]: Started Journal Service. client # [ 8.016200] systemd[1]: Starting Load/Save OS Random Seed... client # [ 8.017110] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.028335] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 8.029539] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 8.051416] systemd-modules-load[268]: Module 'loop' is built in worker2 # [ 8.045024] systemd-modules-load[272]: Module 'atkbd' is built in worker1 # [ 8.060076] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 8.125638] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 8.055648] systemd-modules-load[272]: Module 'loop' is built in worker2 # [ 8.069420] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 8.144861] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.100780] systemd-journald[269]: Received client request to flush runtime journal. lb # [ 8.186107] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 8.125546] systemd-oomd[273]: No swap; memory pressure usage will be degraded lb # [ 8.200375] systemd[1]: Starting Update is Completed... worker2 # [ 8.140270] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 8.146008] systemd[1]: Reached target First Boot Complete. client # [ 8.145640] systemd[1]: Finished Load/Save OS Random Seed. client # [ 8.146657] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.151910] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 8.154655] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.175259] systemd-oomd[270]: No swap; memory pressure usage will be degraded worker1 # [ 8.189776] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 8.190643] systemd[1]: Reached target First Boot Complete. worker1 # [ 8.191442] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 8.210049] systemd-journald[267]: Received client request to flush runtime journal. worker2 # [ 8.192259] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 8.208587] systemd-journald[271]: Received client request to flush runtime journal. client # [ 8.197531] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.309566] systemd[1]: Finished Update is Completed. client # [ 8.238713] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.241042] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.260451] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.261593] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.261490] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.262709] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.276234] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.277330] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 8.276898] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.376188] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.377885] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.382137] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.375452] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.377473] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.388290] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.499982] systemd-udevd[315]: Using default interface naming scheme 'v261'. worker1 # [ 8.517995] systemd-udevd[310]: Using default interface naming scheme 'v261'. worker2 # [ 8.519212] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.548963] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.582146] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.586903] systemd[1]: Reached target Local File Systems. worker2 # [ 8.587767] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.606990] systemd[1]: Starting register-nix-paths.service... worker2 # [ 8.612851] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.628846] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.630829] systemd[1]: Reached target Local File Systems. worker2 # [ 8.620613] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.634304] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.628368] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.629359] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.642491] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.643385] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.671064] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.673906] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.674853] systemd[1]: Starting Create System Files and Directories... client # [ 8.708955] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.711174] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.720217] systemd[1]: Mounting /run/wrappers... client # [ 8.724700] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.762956] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 8.752186] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.761368] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.787879] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.807313] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.816274] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.878311] systemd[1]: Finished Coldplug All udev Devices. client # [ 8.828799] systemd[1]: Mounted /run/wrappers. client # [ 8.829636] systemd[1]: Reached target Local File Systems. client # [ 8.836805] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.842331] systemd[1]: Starting register-nix-paths.service... client # [ 8.850448] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.851356] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.860351] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.873566] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.893686] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.901563] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 8.912437] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 8.941503] systemd[1]: Finished Create System Files and Directories. client # [ 8.927946] systemd-udevd[299]: Using default interface naming scheme 'v261'. worker1 # [ 8.948421] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 8.960620] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.959038] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.974035] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.998337] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 9.038702] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 9.073143] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.082465] systemd[1]: Finished Create System Files and Directories. lb # [ 9.166163] systemd[1]: Finished register-nix-paths.service. client # [ 9.091739] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 9.175861] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.099511] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 9.179696] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.184334] systemd[1]: Reached target System Initialization. lb # [ 9.185240] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.186245] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.187086] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 9.110031] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 9.194653] systemd[1]: Reached target Timer Units. lb # [ 9.195439] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 9.124577] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.203075] systemd[1]: Listening on niks3 server socket. worker2 # [ 9.129459] systemd[1]: Starting Update is Completed... lb # [ 9.203976] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.204945] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 9.144286] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.206108] systemd[1]: Reached target Socket Units. lb # [ 9.206847] systemd[1]: Reached target Basic System. worker1 # [ 9.149519] systemd[1]: Starting Update is Completed... lb # [ 9.216212] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 9.217516] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.227382] systemd[1]: Starting Post-Boot Actions... lb # [ 9.253900] systemd[1]: Started Reset console on configuration changes. lb # [ 9.291884] systemd[1]: Starting resolvconf update... client # [ 9.216566] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 9.229907] systemd[1]: Finished Update is Completed. worker1 # [ 9.251035] systemd[1]: Finished Update is Completed. lb # [ 9.319221] systemd[1]: Started rustfs.service. client # [ 9.255082] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.339466] systemd[1]: Starting rustfs-bucket.service... client # [ 9.261563] systemd[1]: Starting Update is Completed... lb # [ 9.378050] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 9.379427] nsncd[440]: Sep 23 06:59:09.008 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.394053] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.395027] systemd[1]: Reached target User and Group Name Lookups. lb # [ 9.412120] systemd[1]: Starting D-Bus System Message Bus... client # [ 9.344518] systemd[1]: Finished Update is Completed. lb # [ 9.450140] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 9.461845] systemd[1]: Starting User Login Management... lb # [ 9.466635] systemd[1]: Finished Post-Boot Actions. lb # [ 9.586656] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 9.628758] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 9.647635] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.728528] dbus-broker-launch[465]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 9.656439] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.738506] dbus-broker-launch[465]: NSS returned no entry for 'systemd-timesync' lb # [ 9.739608] dbus-broker-launch[465]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 9.704306] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.706277] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.701446] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.807184] systemd-logind[467]: New seat seat0. lb # [ 9.823492] systemd[1]: Started User Login Management. lb # [ 9.842959] systemd[1]: Starting linger-users.service... lb # [ 9.843866] systemd[1]: Started D-Bus System Message Bus. client # [ 9.840795] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.843691] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.928362] dbus-broker-launch[465]: Ready lb # [ 9.958324] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 9.959493] systemd[1]: Stopping Host and Network Name Lookups... client # [ 9.883176] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 9.970967] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 9.971964] systemd[1]: Stopping User and Group Name Lookups... lb # [ 9.981969] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 9.982979] systemd[1]: nscd.service: Deactivated successfully. lb # [ 9.983815] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 9.997444] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 10.002047] systemd[1]: Finished linger-users.service. worker2 # [ 9.943409] systemd[1]: Finished Firewall. lb # [ 10.018599] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 9.939761] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 10.026654] systemd[1]: Finished Firewall. lb # [ 10.126855] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.134729] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.136805] nsncd[546]: Sep 23 06:59:09.769 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.138847] systemd[1]: Reached target User and Group Name Lookups. lb # [ 10.174016] systemd[1]: Finished resolvconf update. lb # [ 10.185582] systemd[1]: Starting DHCP Client... client # [ 10.134221] (udev-worker)[358]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.140643] (udev-worker)[347]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 10.146626] (udev-worker)[347]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.389403] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.384982] systemd[1]: Finished register-nix-paths.service. client # [ 10.389302] systemd[1]: Reached target System Initialization. client # [ 10.392321] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.394052] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.398742] systemd[1]: Reached target Timer Units. client # [ 10.400540] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.481457] dhcpcd[588]: dhcpcd-10.3.2 starting client # [ 10.404197] systemd[1]: Listening on Nix Daemon Socket. client # [ 10.406487] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.413651] systemd[1]: Reached target Socket Units. client # [ 10.414480] systemd[1]: Reached target Basic System. client # [ 10.415990] systemd[1]: Started backdoor.service. lb # [ 10.499133] dhcpcd[603]: dev: loaded udev worker2 # [ 10.425690] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.421317] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.503748] dhcpcd[603]: no valid interfaces found client # [ 10.426563] systemd[1]: Starting nftables firewall... lb # [ 10.506751] dhcpcd[603]: no valid interfaces found lb # [ 10.507455] dhcpcd[603]: libudev: received NULL device lb # [ 10.510829] dhcpcd[603]: libudev: received NULL device client # [ 10.434879] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.469207] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 10.460317] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.477165] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 10.481087] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.486680] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.499839] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 10.489080] systemd[1]: Reached target System Initialization. worker2 # [ 10.490686] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 10.493991] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.500913] systemd[1]: Reached target Timer Units. worker2 # [ 10.505748] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 10.503695] systemd[1]: Starting resolvconf update... worker2 # [ 10.516336] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.531179] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.532443] systemd[1]: Reached target System Initialization. worker1 # [ 10.533338] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.534836] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.537741] systemd[1]: Reached target Timer Units. worker2 # [ 10.528530] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 10.544209] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.538952] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.551757] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.546710] systemd[1]: Reached target Socket Units. worker2 # [ 10.550048] systemd[1]: Reached target Basic System. worker2 # [ 10.555241] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.567625] systemd[1]: Listening on Nix worker-protocol over gRPC. client # [ 10.554014] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.564526] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.577336] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # connecting to host... worker2 # [ 10.575721] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.588156] systemd[1]: Reached target Socket Units. worker1 # [ 10.591183] systemd[1]: Reached target Basic System. worker1 # [ 10.595169] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.588126] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.601213] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.610523] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.599169] systemd[1]: Starting resolvconf update... worker2 # [ 10.605334] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.617992] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.614870] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 10.689681] systemd[1]: Starting Virtual Console Setup... lb # [ 10.691055] systemd[1]: Finished Firewall. worker1 # [ 10.633015] systemd[1]: Starting resolvconf update... client # [ 10.621015] systemd[1]: Starting D-Bus System Message Bus... lb # [ 10.703900] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.643034] systemd[1]: Starting D-Bus System Message Bus... client: Guest shell says: b'Spawning backdoor root shell...\n' client # [ 10.633941] nsncd[393]: Sep 23 06:59:10.280 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.719066] systemd[1]: Starting Extra networking commands.... worker2 # [ 10.651693] systemd[1]: Finished Post-Boot Actions. client # [ 10.649224] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.650140] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.692444] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.684762] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.688814] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.681370] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.778408] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 10.698317] nsncd[488]: Sep 23 06:59:10.319 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.723807] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.709228] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.682324] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.714795] systemd[1]: Starting User Login Management... client: connected to guest root shell client # [ 10.711846] systemd[1]: Starting User Login Management... worker1 # [ 10.730471] nsncd[485]: Sep 23 06:59:10.347 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.735477] systemd[1]: Finished Import lastlog data into lastlog2 database. client: (connecting took 11.18 seconds) client # [ 10.721006] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.747872] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.747648] dbus-broker-launch[492]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.752220] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.755303] dbus-broker-launch[492]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.756946] systemd[1]: Starting User Login Management... worker2 # [ 10.760563] dbus-broker-launch[492]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.781768] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.775623] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.797781] dbus-broker-launch[489]: Looking up NSS user entry for 'systemd-timesync'... client: (finished: waiting for the VM to finish booting, in 11.18 seconds) worker1 # [ 10.807246] dbus-broker-launch[489]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.812763] dbus-broker-launch[489]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.826777] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.826438] dbus-broker-launch[492]: Ready worker1 # [ 10.875639] dbus-broker-launch[489]: Ready client # [ 10.873585] dbus-broker-launch[406]: Looking up NSS user entry for 'systemd-timesync'... client # [ 10.895713] dbus-broker-launch[406]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.901914] systemd-logind[500]: New seat seat0. client # [ 10.899593] dbus-broker-launch[406]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.909045] systemd[1]: Started User Login Management. worker2 # [ 10.917746] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.925481] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 10.937783] systemd-logind[497]: New seat seat0. worker1 # [ 10.942595] systemd[1]: Started User Login Management. worker2 # [ 10.938318] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.951206] systemd[1]: Starting linger-users.service... worker2 # [ 10.946193] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 10.952624] systemd[1]: Starting linger-users.service... worker1 # [ 10.965020] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.955249] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.958113] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 10.972653] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 10.964179] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 10.977843] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.978905] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.979754] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.969323] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.997972] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 11.004966] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 11.009710] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.989136] systemd-logind[422]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 10.997972] systemd-logind[422]: New seat seat0. client # [ 10.998680] systemd[1]: Started D-Bus System Message Bus. client # [ 11.015625] systemd[1]: Started User Login Management. worker2 # [ 11.024512] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.019937] systemd[1]: Starting linger-users.service... worker2 # [ 11.029367] systemd[1]: Finished linger-users.service. worker2 # [ 11.041023] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.056487] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 11.045317] nsncd[555]: Sep 23 06:59:10.673 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 11.060273] systemd[1]: Finished linger-users.service. worker2 # [ 11.056148] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.052527] dbus-broker-launch[406]: Ready worker2 # [ 11.057940] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.070473] mousedev: PS/2 mouse device common for all mice lb # [ 11.145124] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.076363] systemd[1]: Finished resolvconf update. lb # [ 11.150045] systemd[1]: Reached target Network. worker1 # [ 11.091741] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 11.080689] systemd[1]: Reached target Preparation for Network. worker1 # [ 11.095786] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 11.086800] systemd[1]: Starting DHCP Client... lb # [ 11.161243] systemd[1]: Starting PostgreSQL Server... worker1 # [ 11.100444] nsncd[552]: Sep 23 06:59:10.718 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 11.090910] systemd[1]: Starting Extra networking commands.... lb # [ 11.167529] systemd[1]: Starting Permit User Sessions... worker1 # [ 11.107987] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.111864] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 11.132424] systemd[1]: Finished resolvconf update. worker1 # [ 11.134411] systemd[1]: Reached target Preparation for Network. client # [ 11.118157] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.119157] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 11.139540] systemd[1]: Starting DHCP Client... worker2 # [ 11.130052] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 11.145381] systemd[1]: Starting Extra networking commands.... client # [ 11.119985] systemd[1]: Stopped target User and Group Name Lookups. client # [ 11.134008] systemd[1]: Stopping User and Group Name Lookups... client # [ 11.134944] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 11.142732] systemd[1]: nscd.service: Deactivated successfully. client # [ 11.143633] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 11.155754] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.160280] systemd[1]: Finished linger-users.service. worker1 # [ 11.199285] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 11.185166] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 11.206819] systemd[1]: Started backdoor.service. lb # [ 11.268682] systemd[1]: Finished Permit User Sessions. lb # [ 11.280185] systemd[1]: Started Getty on tty1. lb # [ 11.283597] systemd[1]: Reached target Login Prompts. worker2 # [ 11.256673] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 11.264265] systemd[1]: Started backdoor.service. client # [ 11.274061] systemd[1]: Finished resolvconf update. client # [ 11.291208] nsncd[492]: Sep 23 06:59:10.932 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 11.373256] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 11.296602] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.297711] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.298569] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.299400] systemd-logind[422]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # connecting to host... client # [ 11.312417] systemd[1]: Starting DHCP Client... lb # [ 11.436546] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 11.444120] systemd[1]: Started backdoor.service. worker2 # connecting to host... worker2 # [ 11.378165] dhcpcd[590]: dhcpcd-10.3.2 starting worker1 # [ 11.406572] (udev-worker)[358]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 11.398313] dhcpcd[634]: dev: loaded udev worker1 # [ 11.416565] (udev-worker)[358]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.432928] (udev-worker)[343]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.445939] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.464570] dhcpcd[588]: dhcpcd-10.3.2 starting worker1 # [ 11.475053] dhcpcd[635]: dev: loaded udev lb # [ 11.560818] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.553766] postgresql-pre-start[692]: The files belonging to this database system will be owned by user "postgres". lb # [ 11.555224] postgresql-pre-start[692]: This user must also own the server process. worker2 # [ 11.481953] systemd[1]: Finished Extra networking commands.. lb # connecting to host... lb # [ 11.566903] postgresql-pre-start[692]: The database cluster will be initialized with locale "en_US.UTF-8". worker2 # [ 11.496857] systemd[1]: Reached target Network. lb # [ 11.571411] postgresql-pre-start[692]: The default database encoding has accordingly been set to "UTF8". worker2 # [ 11.500231] systemd[1]: Starting Permit User Sessions... worker1 # [ 11.524188] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.576945] postgresql-pre-start[692]: The default text search configuration will be set to "english". lb # [ 11.578163] postgresql-pre-start[692]: Data page checksums are enabled. lb # [ 11.579020] postgresql-pre-start[692]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok client # [ 11.502961] dhcpcd[520]: dhcpcd-10.3.2 starting lb # [ 11.588570] postgresql-pre-start[692]: creating subdirectories ... ok client # [ 11.513063] dhcpcd[528]: dev: loaded udev worker2 # [ 11.519077] (udev-worker)[365]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 11.594626] postgresql-pre-start[692]: selecting dynamic shared memory implementation ... posix worker1 # [ 11.536502] systemd[1]: Finished Extra networking commands.. worker1 # [ 11.537391] systemd[1]: Reached target Network. worker2 # [ 11.530353] (udev-worker)[365]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.531933] (udev-worker)[362]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.546028] systemd[1]: Starting Permit User Sessions... lb # [ 11.606577] (udev-worker)[663]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 11.618052] (udev-worker)[638]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.564171] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.625644] (udev-worker)[663]: Network interface NamePolicy= disabled on kernel command line. client # [ 11.566338] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.659599] systemd-vconsole-setup[622]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 11.605708] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.607213] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.610562] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.610899] cfg80211: failed to load regulatory.db lb # [ 11.676668] systemd[1]: Finished Virtual Console Setup. worker2 # [ 11.604093] systemd[1]: Finished Permit User Sessions. worker2 # [ 11.608969] systemd[1]: Started Getty on tty1. worker2 # [ 11.609676] systemd[1]: Reached target Login Prompts. worker1 # [ 11.641760] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.638649] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.655920] systemd[1]: Started Getty on tty1. worker1 # [ 11.657358] systemd[1]: Reached target Login Prompts. worker1 # [ 11.658088] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.648669] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.661346] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.656224] systemd[1]: Starting Address configuration of eth1... lb # [ 11.745872] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.696007] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.698052] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.682967] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.702344] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.702687] cfg80211: failed to load regulatory.db worker2 # [ 11.709455] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.698997] dhcpcd[634]: eth0: waiting for carrier worker2 # [ 11.702675] dhcpcd[634]: eth0: waiting for carrier worker2 # [ 11.706838] dhcpcd[634]: eth0: carrier acquired worker2 # [ 11.730482] dhcpcd[634]: DUID 00:01:00:01:32:46:35:3f:52:54:00:12:34:56 worker2 # [ 11.731475] dhcpcd[634]: eth0: IAID 00:12:34:56 client # [ 11.743538] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.744078] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.736320] dhcpcd[634]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.746689] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.747018] cfg80211: failed to load regulatory.db lb # [ 11.813130] postgresql-pre-start[692]: selecting default "max_connections" ... 100 lb # [ 11.833743] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.834330] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.762873] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.775796] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.837578] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.837889] cfg80211: failed to load regulatory.db worker2 # [ 11.765846] network-addresses-eth1-start[661]: adding address 192.168.1.4/24... done worker1 # [ 11.792733] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.782809] dhcpcd[635]: eth0: waiting for carrier worker1 # [ 11.787487] dhcpcd[635]: eth0: waiting for carrier worker2 # [ 11.777983] network-addresses-eth1-start[661]: adding address 2001:db8:1::4/64... done worker1 # [ 11.790470] dhcpcd[635]: eth0: carrier acquired worker1 # [ 11.791491] network-addresses-eth1-start[653]: adding address 192.168.1.3/24... done client # [ 11.781796] systemd[1]: Finished nftables firewall. client # [ 11.785458] systemd[1]: Reached target Preparation for Network. worker1 # [ 11.806166] network-addresses-eth1-start[653]: adding address 2001:db8:1::3/64... done worker2 # [ 11.795760] systemd[1]: Finished Address configuration of eth1. client # [ 11.794229] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.811702] dhcpcd[635]: DUID 00:01:00:01:32:46:35:3f:52:54:00:12:34:56 worker1 # [ 11.815495] dhcpcd[635]: eth0: IAID 00:12:34:56 worker1 # [ 11.817245] dhcpcd[635]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.827337] systemd[1]: Finished Address configuration of eth1. client # [ 11.812892] systemd[1]: Starting Extra networking commands.... client # [ 11.864966] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.854041] dhcpcd[528]: eth0: waiting for carrier client # [ 11.857957] dhcpcd[528]: libudev: received NULL device client # [ 11.858740] dhcpcd[528]: libudev: received NULL device client # [ 11.859635] dhcpcd[528]: eth0: carrier acquired client # [ 11.879342] dhcpcd[528]: DUID 00:01:00:01:32:46:35:3f:52:54:00:12:34:56 lb # [ 11.960433] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 11.882863] dhcpcd[528]: eth0: IAID 00:12:34:56 client # [ 11.883652] dhcpcd[528]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.971004] systemd[1]: Starting Address configuration of eth1... lb # [ 11.989778] postgresql-pre-start[692]: selecting default "shared_buffers" ... 128MB client # [ 11.928863] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.939431] network-addresses-eth1-start[546]: adding address 192.168.1.1/24... done worker2 # [ 11.953787] systemd-logind[500]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.961763] network-addresses-eth1-start[546]: adding address 2001:db8:1::1/64... done worker1 # [ 11.990875] systemd-logind[497]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.987268] systemd[1]: Finished Address configuration of eth1. lb # [ 12.083240] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 12.075206] dhcpcd[603]: eth0: waiting for carrier lb # [ 12.078388] dhcpcd[603]: eth0: carrier acquired lb # [ 12.102270] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 12.099608] dhcpcd[603]: DUID 00:01:00:01:32:46:35:3f:52:54:00:12:34:56 lb # [ 12.103039] dhcpcd[603]: eth0: IAID 00:12:34:56 lb # [ 12.103704] dhcpcd[603]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.122390] network-addresses-eth1-start[716]: adding address 192.168.1.2/24... done lb # [ 12.131512] systemd-logind[467]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 12.059032] dhcpcd[528]: eth0: soliciting a DHCP lease client # [ 12.064637] dhcpcd[528]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 12.145458] network-addresses-eth1-start[716]: adding address 2001:db8:1::2/64... done client # [ 12.072238] dhcpcd[528]: eth0: probing address 10.0.2.15/24 worker2 # [ 12.108438] mousedev: PS/2 mouse device common for all mice worker1 # [ 12.120523] mousedev: PS/2 mouse device common for all mice client # [ 12.094014] systemd[1]: Finished Extra networking commands.. lb # [ 12.173238] systemd[1]: Finished Address configuration of eth1. client # [ 12.099382] systemd[1]: Reached target Network. client # [ 12.102964] systemd[1]: Starting Permit User Sessions... worker2 # [ 12.160668] systemd-logind[500]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 12.161188] systemd[1]: Finished Permit User Sessions. client # [ 12.168524] systemd[1]: Started Getty on tty1. client # [ 12.171352] systemd[1]: Reached target Login Prompts. worker1 # [ 12.205567] systemd-logind[497]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 12.306407] mousedev: PS/2 mouse device common for all mice lb # [ 12.322291] systemd-logind[467]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client: (finished: waiting for unit network-addresses-eth1.service, in 12.68 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.04 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb: (finished: waiting for unit network-addresses-eth1.service, in 0.10 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 12.636311] dhcpcd[603]: eth0: soliciting a DHCP lease worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.09 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } lb # [ 12.644574] dhcpcd[603]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 12.652222] dhcpcd[603]: eth0: probing address 10.0.2.15/24 client # [ 12.603325] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.04 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.04 seconds) lb: waiting for unit niks3.service worker1 # [ 12.851497] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2 # [ 12.843731] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.920075] systemd[1]: Starting Virtual Console Setup... client # [ 12.945668] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 12.959919] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.962816] systemd[1]: Stopped Virtual Console Setup. client # [ 12.972456] systemd[1]: Starting Virtual Console Setup... client # [ 13.007126] systemd-logind[422]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.083464] dhcpcd[635]: eth0: soliciting a DHCP lease worker1 # [ 13.093386] dhcpcd[635]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 13.100214] dhcpcd[635]: eth0: probing address 10.0.2.15/24 client # [ 13.102041] systemd-vconsole-setup[638]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 13.105730] systemd[1]: Finished Virtual Console Setup. worker2 # [ 13.123296] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.144633] dhcpcd[634]: eth0: soliciting a DHCP lease worker1 # [ 13.159588] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.152632] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.165697] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.154922] dhcpcd[634]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 13.163805] dhcpcd[634]: eth0: probing address 10.0.2.15/24 worker2 # [ 13.167762] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.174728] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.175474] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.190224] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 13.191273] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.203741] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.231936] systemd-logind[500]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.249579] systemd-logind[497]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.307581] systemd-vconsole-setup[702]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.311522] systemd[1]: Finished Virtual Console Setup. worker1 # [ 13.333646] systemd-vconsole-setup[701]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 13.337575] systemd[1]: Finished Virtual Console Setup. client # [ 13.529153] dhcpcd[528]: eth0: soliciting an IPv6 router client # [ 13.531511] dhcpcd[528]: eth0: Router Advertisement from fe80::2 client # [ 13.534202] dhcpcd[528]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 13.537054] dhcpcd[528]: eth0: adding route to fec0::/64 client # [ 13.539239] dhcpcd[528]: eth0: adding default route via fe80::2 lb # [ 13.689624] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.691712] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.691803] systemd[1]: Stopping Virtual Console Setup... lb # [ 13.698891] systemd[1]: Starting Virtual Console Setup... lb # [ 13.720647] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.723084] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.741645] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.750356] systemd[1]: Starting Virtual Console Setup... lb # [ 13.786483] systemd-logind[467]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 13.998234] dhcpcd[603]: eth0: soliciting an IPv6 router lb # [ 13.999444] dhcpcd[603]: eth0: Router Advertisement from fe80::2 lb # [ 14.002590] dhcpcd[603]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.003493] dhcpcd[603]: eth0: adding route to fec0::/64 lb # [ 14.008799] dhcpcd[603]: eth0: adding default route via fe80::2 lb # [ 14.013114] systemd-vconsole-setup[758]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 14.021217] systemd[1]: Finished Virtual Console Setup. worker1 # [ 14.141765] dhcpcd[635]: eth0: soliciting an IPv6 router worker1 # [ 14.145118] dhcpcd[635]: eth0: Router Advertisement from fe80::2 worker1 # [ 14.147662] dhcpcd[635]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 14.150818] dhcpcd[635]: eth0: adding route to fec0::/64 lb # [ 14.214852] postgresql-pre-start[692]: selecting default time zone ... UTC worker1 # [ 14.153151] dhcpcd[635]: eth0: adding default route via fe80::2 lb # [ 14.217525] postgresql-pre-start[692]: creating configuration files ... ok lb # [ 14.435226] postgresql-pre-start[692]: running bootstrap script ... ok worker2 # [ 14.397851] dhcpcd[634]: eth0: soliciting an IPv6 router worker2 # [ 14.401217] dhcpcd[634]: eth0: Router Advertisement from fe80::2 worker2 # [ 14.403721] dhcpcd[634]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 14.406830] dhcpcd[634]: eth0: adding route to fec0::/64 worker2 # [ 14.409195] dhcpcd[634]: eth0: adding default route via fe80::2 lb # [ 14.906294] postgresql-pre-start[692]: performing post-bootstrap initialization ... ok lb # [ 15.103063] postgresql-pre-start[692]: syncing data to disk ... ok lb # [ 15.103996] postgresql-pre-start[692]: initdb: warning: enabling "trust" authentication for local connections lb # [ 15.107342] postgresql-pre-start[692]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 15.113409] postgresql-pre-start[692]: Success. You can now start the database server using: lb # [ 15.114508] postgresql-pre-start[692]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 15.226117] postgres[786]: [786] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 15.228994] postgres[786]: [786] LOG: listening on IPv6 address "::1", port 5432 lb # [ 15.230167] postgres[786]: [786] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 15.232130] postgres[786]: [786] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 15.244423] postgres[796]: [796] LOG: database system was shut down at 2026-09-23 06:59:14 GMT lb # [ 15.249690] postgres[786]: [786] LOG: database system is ready to accept connections lb # [ 15.253431] systemd[1]: Started PostgreSQL Server. lb # [ 15.259038] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 15.419697] postgresql-setup-start[807]: CREATE DATABASE lb # [ 15.455312] postgresql-setup-start[812]: CREATE ROLE lb # [ 15.470751] postgresql-setup-start[814]: ALTER DATABASE lb # [ 15.476285] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 15.478215] systemd[1]: Reached target PostgreSQL. client # [ 17.175246] dhcpcd[528]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 17.178198] dhcpcd[528]: eth0: adding route to 10.0.2.0/24 client # [ 17.180887] dhcpcd[528]: eth0: adding default route via 10.0.2.2 client # [ 17.352230] systemd[1]: Started DHCP Client. client # [ 17.354121] systemd[1]: Reached target Multi-User System. client # [ 17.355232] systemd[1]: Startup finished in 919ms (kernel) + 4.963s (initrd) + 11.471s (userspace) = 17.354s. lb # [ 17.530323] dhcpcd[603]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.533431] dhcpcd[603]: eth0: adding route to 10.0.2.0/24 lb # [ 17.533595] dhcpcd[603]: eth0: adding default route via 10.0.2.2 lb # [ 17.670251] systemd[1]: Started DHCP Client. lb # [ 17.677189] systemd[1]: Reached target Network is Online. lb # [ 17.680511] systemd[1]: Started Envoy reverse proxy. lb # [ 17.688569] systemd[1]: Started mock-oidc.service. worker1 # [ 17.837922] dhcpcd[635]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 17.840893] dhcpcd[635]: eth0: adding route to 10.0.2.0/24 worker1 # [ 17.843184] dhcpcd[635]: eth0: adding default route via 10.0.2.2 lb # [ 17.909182] mock-oidc-server[896]: Mock OIDC Server running lb # [ 17.910839] mock-oidc-server[896]: OIDC Address: 192.168.1.2:8080 lb # [ 17.912308] mock-oidc-server[896]: Issue Address: 0.0.0.0:8081 lb # [ 17.913614] mock-oidc-server[896]: Issuer: http://192.168.1.2:8080/oidc lb # [ 17.914868] mock-oidc-server[896]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 17.916342] mock-oidc-server[896]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 17.917931] mock-oidc-server[896]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 18.038721] envoy[895]: [2026-09-23 06:59:17.673][895][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 18.041591] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:469] statically linked extensions: worker1 # [ 17.981779] systemd[1]: Started DHCP Client. worker1 # [ 17.984130] systemd[1]: Reached target Multi-User System. lb # [ 18.043030] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection worker1 # [ 17.985151] systemd[1]: Startup finished in 946ms (kernel) + 5.041s (initrd) + 11.995s (userspace) = 17.983s. lb # [ 18.048480] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 18.050372] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 18.052653] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 18.055036] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 18.057522] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 18.063710] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 18.066059] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 18.068786] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 18.071722] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 18.096145] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 18.099097] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 18.103472] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.107065] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 18.115223] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 18.117576] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 18.119237] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 18.121996] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 18.127019] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 18.129774] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 18.133635] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 18.135965] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 18.143570] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 18.145852] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.148757] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 18.150363] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 18.152473] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 18.154851] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 18.156550] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 18.159657] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 18.162499] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 18.164435] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 18.166876] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 18.169338] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 18.171472] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 18.173332] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 18.176782] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 18.179615] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 18.182386] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.184453] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.186512] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 18.188850] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 18.190981] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 18.193481] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 18.205409] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 18.239185] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 18.241789] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 18.243717] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 18.247333] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 18.250603] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 18.252767] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 18.254567] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 18.256716] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 18.258490] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file worker1 # [ 18.199280] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.261518] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 18.264950] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 18.272137] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 18.274620] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 18.281013] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 18.283298] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 18.286090] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 18.288645] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 18.290403] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 18.293032] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 18.296216] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 18.298417] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 18.299980] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 18.302948] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 18.304643] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 18.306780] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 18.311139] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 18.313083] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 18.315078] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 18.321184] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 18.323148] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 18.324854] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 18.326685] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 18.329688] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 18.332426] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 18.337147] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 18.338949] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 18.340595] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 18.342363] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 18.349383] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 18.351984] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 18.353880] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 18.359835] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 18.373297] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 18.375764] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 18.377713] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 18.382349] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 18.384267] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 18.386167] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 18.390807] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 18.393262] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 18.395686] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 18.405419] envoy[895]: [2026-09-23 06:59:17.676][895][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 18.425833] envoy[895]: [2026-09-23 06:59:17.677][895][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 18.427399] envoy[895]: [2026-09-23 06:59:17.677][895][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 18.430788] envoy[895]: [2026-09-23 06:59:17.677][895][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 18.432730] envoy[895]: [2026-09-23 06:59:17.677][895][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder worker1 # [ 18.377668] nix-grpc-daemon[765]: ts=2026-09-23T06:59:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.442687] envoy[895]: [2026-09-23 06:59:17.677][895][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd worker1 # [ 18.385770] nix-grpc-daemon[765]: ts=2026-09-23T06:59:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 18.387724] nix-grpc-daemon[765]: ts=2026-09-23T06:59:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.452467] envoy[895]: [2026-09-23 06:59:17.677][895][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 18.455085] envoy[895]: [2026-09-23 06:59:17.677][895][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 18.459121] envoy[895]: [2026-09-23 06:59:17.698][895][info][main] [source/server/server.cc:531] HTTP header map info: worker1 # [ 18.402679] nix-grpc-daemon[765]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 18.406176] nix-grpc-daemon[765]: E0000 00:00:1790146758.026092 765 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.460864] envoy[895]: [2026-09-23 06:59:17.699][895][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id worker1 # [ 18.413654] nix-grpc-daemon[765]: E0000 00:00:1790146758.035414 765 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.476533] envoy[895]: [2026-09-23 06:59:17.699][895][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker1 # [ 18.415927] nix-grpc-daemon[765]: ts=2026-09-23T06:59:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket lb # [ 18.478022] envoy[895]: [2026-09-23 06:59:17.699][895][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 18.487323] envoy[895]: [2026-09-23 06:59:17.699][895][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 18.492404] envoy[895]: [2026-09-23 06:59:17.730][895][info][main] [source/server/server.cc:968] runtime: {} lb # [ 18.494150] envoy[895]: [2026-09-23 06:59:17.738][895][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 18.495809] envoy[895]: [2026-09-23 06:59:17.738][895][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 18.497875] envoy[895]: [2026-09-23 06:59:17.738][895][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 18.499360] envoy[895]: [2026-09-23 06:59:17.739][895][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 18.501289] envoy[895]: [2026-09-23 06:59:17.739][895][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 18.502849] envoy[895]: [2026-09-23 06:59:17.739][895][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 18.504495] envoy[895]: [2026-09-23 06:59:17.885][895][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 18.506389] envoy[895]: [2026-09-23 06:59:17.899][895][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 18.509461] envoy[895]: [2026-09-23 06:59:17.900][895][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 18.513076] envoy[895]: [2026-09-23 06:59:17.900][895][info][main] [source/server/server.cc:1077] starting main dispatch loop lb # [ 18.514532] envoy[895]: [2026-09-23 06:59:18.144][895][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 18.517336] envoy[895]: [2026-09-23 06:59:18.144][895][info][upstream] [source/common/upstream/cluster_manager_impl.cc:227] cm init: initializing secondary clusters worker1 # [ 18.435751] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 18.472626] nix-grpc-daemon[765]: E0000 00:00:1790146758.064643 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.474928] nix-grpc-daemon[765]: E0000 00:00:1790146758.082727 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.490375] nix-grpc-daemon[765]: E0000 00:00:1790146758.083259 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.493443] nix-grpc-daemon[765]: E0000 00:00:1790146758.106285 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.495821] nix-grpc-daemon[765]: ts=2026-09-23T06:59:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.634002] dhcpcd[634]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 18.637016] dhcpcd[634]: eth0: adding route to 10.0.2.0/24 worker2 # [ 18.639311] dhcpcd[634]: eth0: adding default route via 10.0.2.2 worker1 # [ 18.685776] nix-grpc-daemon[765]: ts=2026-09-23T06:59:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.778503] systemd[1]: Started DHCP Client. worker2 # [ 18.782248] systemd[1]: Reached target Multi-User System. worker2 # [ 18.783256] systemd[1]: Reached target Network is Online. worker2 # [ 18.788195] systemd[1]: Starting scheduler-token.service... worker2 # [ 18.840238] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 18.842625] systemd[1]: Finished scheduler-token.service. worker2 # [ 18.843647] systemd[1]: scheduler-token.service: Consumed 28ms CPU time over 56ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 18.857803] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 18.966916] nix-grpc-daemon[770]: ts=2026-09-23T06:59:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 18.975933] nix-grpc-daemon[770]: ts=2026-09-23T06:59:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 18.978591] nix-grpc-daemon[770]: ts=2026-09-23T06:59:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 18.992124] nix-grpc-daemon[770]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 18.994994] nix-grpc-daemon[770]: E0000 00:00:1790146758.623871 770 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.999779] nix-grpc-daemon[770]: E0000 00:00:1790146758.632034 770 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 19.014749] nix-grpc-daemon[770]: ts=2026-09-23T06:59:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 19.019821] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 19.028111] systemd[1]: Startup finished in 935ms (kernel) + 4.989s (initrd) + 13.079s (userspace) = 19.004s. worker2 # [ 19.029488] nix-grpc-daemon[770]: E0000 00:00:1790146758.654175 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 19.043732] nix-grpc-daemon[770]: E0000 00:00:1790146758.675799 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 19.062536] nix-grpc-daemon[770]: E0000 00:00:1790146758.678509 812 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 19.065810] nix-grpc-daemon[770]: E0000 00:00:1790146758.693905 812 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 19.068602] nix-grpc-daemon[770]: ts=2026-09-23T06:59:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 19.086437] nix-grpc-daemon[765]: ts=2026-09-23T06:59:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.269267] nix-grpc-daemon[770]: ts=2026-09-23T06:59:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.669929] nix-grpc-daemon[770]: ts=2026-09-23T06:59:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 19.888215] nix-grpc-daemon[765]: ts=2026-09-23T06:59:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.471723] nix-grpc-daemon[770]: ts=2026-09-23T06:59:20Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 21.488796] nix-grpc-daemon[765]: ts=2026-09-23T06:59:21Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 22.072196] nix-grpc-daemon[770]: ts=2026-09-23T06:59:21Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 24.689339] nix-grpc-daemon[765]: ts=2026-09-23T06:59:24Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 25.272755] nix-grpc-daemon[770]: ts=2026-09-23T06:59:24Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 28.864842] rustfs-bucket-start[1004]: mb s3://farm lb # [ 28.877922] systemd[1]: Finished rustfs-bucket.service. lb # [ 28.888219] systemd[1]: Starting niks3 server... lb # [ 29.023935] postgres[1020]: [1020] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 29.028191] postgres[1020]: [1020] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 29.050996] niks3-server[1015]: 2026/09/23 06:59:28 OK 20241026095416_initial_model.sql (15.68ms) lb # [ 29.054096] niks3-server[1015]: 2026/09/23 06:59:28 OK 20251210153512_drop_unused_gin_index.sql (3.37ms) lb # [ 29.059225] niks3-server[1015]: 2026/09/23 06:59:28 OK 20251218171726_add_pins.sql (5.07ms) lb # [ 29.063733] niks3-server[1015]: 2026/09/23 06:59:28 OK 20260628120000_add_object_size_and_stats.sql (4.47ms) lb # [ 29.068836] niks3-server[1015]: 2026/09/23 06:59:28 OK 20260905000000_add_claims.sql (5.07ms) lb # [ 29.072359] niks3-server[1015]: 2026/09/23 06:59:28 OK 20260920000000_drop_claims.sql (3.42ms) lb # [ 29.074591] niks3-server[1015]: 2026/09/23 06:59:28 goose: successfully migrated database to version: 20260920000000 lb # [ 29.077554] niks3-server[1015]: 2026/09/23 06:59:28 OK 1_commit_pending_closure.sql (5.16ms) lb # [ 29.080055] niks3-server[1015]: 2026/09/23 06:59:28 OK 2_object_stats_trigger.sql (2.34ms) lb # [ 29.081930] niks3-server[1015]: 2026/09/23 06:59:28 goose: up to current file version: 2 lb # [ 29.083708] niks3-server[1015]: 2026/09/23 06:59:28 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 29.107329] niks3-server[1015]: 2026/09/23 06:59:28 INFO Read proxy enabled — serving cache objects from S3 lb # [ 29.113114] niks3-server[1015]: 2026/09/23 06:59:28 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 29.114514] niks3-server[1015]: 2026/09/23 06:59:28 INFO systemd watchdog enabled interval=15s lb # [ 29.115610] niks3-server[1015]: 2026/09/23 06:59:28 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 29.117580] systemd[1]: Started niks3 server. lb # [ 29.118550] systemd[1]: Reached target Multi-User System. lb # [ 29.119461] systemd[1]: Startup finished in 933ms (kernel) + 4.092s (initrd) + 24.085s (userspace) = 29.111s. lb: (finished: waiting for unit niks3.service, in 16.82 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.04 seconds) lb: waiting for TCP port 50051 on localhost worker1 # [ 29.690028] nix-grpc-daemon[765]: ts=2026-09-23T06:59:29Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 30.273368] nix-grpc-daemon[770]: ts=2026-09-23T06:59:29Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 33.513309] envoy[895]: [2026-09-23 06:59:33.147][895][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment lb # [ 33.513551] envoy[895]: [2026-09-23 06:59:33.147][895][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 33.513655] envoy[895]: [2026-09-23 06:59:33.147][895][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 33.538273] envoy[895]: [2026-09-23 06:59:33.172][895][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 33.945945] envoy[895]: conn peer=[::1]:50544 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=10 lb: (finished: waiting for TCP port 50051 on localhost, in 4.22 seconds) worker2 # [ 34.039358] systemd[1]: Starting scheduler-token.service... worker2 # [ 34.105123] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 34.111225] systemd[1]: Finished scheduler-token.service. worker2 # [ 34.112807] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 70ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 34.690582] nix-grpc-daemon[765]: ts=2026-09-23T06:59:34Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2 # [ 35.274954] nix-grpc-daemon[770]: ts=2026-09-23T06:59:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.280172] nix-grpc-daemon[770]: ts=2026-09-23T06:59:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.481558] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.486693] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.687710] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.692914] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.893712] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.898754] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.099653] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.104756] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 36.307362] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.311507] nix-grpc-daemon[770]: ts=2026-09-23T06:59:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 36.508113] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.513023] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.713761] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.718783] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.919791] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.924899] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.125609] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.130594] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 37.332867] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.337358] nix-grpc-daemon[770]: ts=2026-09-23T06:59:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 37.538385] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.543510] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.744460] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.749442] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.950364] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.955356] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.156537] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.163154] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.364082] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.368222] nix-grpc-daemon[770]: ts=2026-09-23T06:59:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 38.565239] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.570284] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.771172] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.776322] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.977277] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.983328] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.055037] nix-grpc-daemon[765]: ts=2026-09-23T06:59:38Z level=info event=scheduler_take_over lb # [ 39.117381] niks3-server[1015]: 2026/09/23 06:59:38 INFO lead: acquired remote=192.168.1.3:59182 worker2 # [ 39.183410] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.188549] nix-grpc-daemon[770]: ts=2026-09-23T06:59:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 39.391456] nix-grpc-daemon[770]: ts=2026-09-23T06:59:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.396900] nix-grpc-daemon[770]: ts=2026-09-23T06:59:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 39.592255] nix-grpc-daemon[770]: ts=2026-09-23T06:59:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.628165] nix-grpc-daemon[765]: ts=2026-09-23T06:59:39Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 39.633613] nix-grpc-daemon[765]: ts=2026-09-23T06:59:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53432 worker1 # [ 39.638883] nix-grpc-daemon[765]: ts=2026-09-23T06:59:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 39.690558] nix-grpc-daemon[765]: ts=2026-09-23T06:59:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.696201] nix-grpc-daemon[765]: ts=2026-09-23T06:59:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53432 worker1 # [ 39.700929] nix-grpc-daemon[765]: ts=2026-09-23T06:59:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker1 # [ 48.450203] nix-grpc-daemon[765]: E0000 00:00:1790146788.068758 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.502950] nix-grpc-daemon[765]: E0000 00:00:1790146788.126211 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 48.544703] nix-grpc-daemon[765]: E0000 00:00:1790146788.132844 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.548866] nix-grpc-daemon[765]: E0000 00:00:1790146788.165237 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 49.029661] nix-grpc-daemon[770]: E0000 00:00:1790146788.658483 810 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 49.084104] nix-grpc-daemon[770]: E0000 00:00:1790146788.714759 810 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 49.115511] nix-grpc-daemon[770]: E0000 00:00:1790146788.721752 810 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 49.119085] nix-grpc-daemon[770]: E0000 00:00:1790146788.746202 810 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 58.129541] nix-grpc-daemon[765]: ts=2026-09-23T06:59:57Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:49168 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 24.71 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.74 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 59.889338] nix-grpc-daemon[765]: ts=2026-09-23T06:59:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 59.897549] systemd[1]: Started Nix Daemon. worker1 # [ 60.096503] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 60.117313] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5) worker1 # [ 60.145812] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 2/5) worker1 # [ 60.341765] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 381 ms (attempt 3/5) worker1 # [ 60.725062] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 130 ms (attempt 4/5) worker1 # [ 60.858093] nix-grpc-daemon[765]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 60.863096] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 60.885010] nix-grpc-daemon[770]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 60.902494] systemd[1]: Started Nix Daemon. worker2 # [ 61.082558] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 61.107674] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5) worker2 # [ 61.164374] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 2/5) worker2 # [ 61.197289] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 339 ms (attempt 3/5) worker2 # [ 61.546183] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 519 ms (attempt 4/5) worker2 # [ 62.077212] nix-grpc-daemon[770]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 62.083488] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 62.095346] nix-grpc-daemon[770]: ts=2026-09-23T07:00:01Z level=debug event=path_info_miss path=1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test error="error: path '/nix/store/1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 62.109338] nix-grpc-daemon[770]: ts=2026-09-23T07:00:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 62.122001] nix-grpc-daemon[770]: ts=2026-09-23T07:00:01Z level=debug event=path_info_miss path=1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test error="error: path '/nix/store/1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker1 # [ 62.465111] nix-grpc-daemon[765]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:49186 worker1 # [ 62.484919] nix-grpc-daemon[765]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:49186 worker1 # [ 62.499880] nix-grpc-daemon[765]: ts=2026-09-23T07:00:02Z level=debug event=path_info_miss path=1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test error="error: path '/nix/store/1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 62.500146] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 62.507233] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:43424 worker2 # [ 62.515462] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=debug event=path_info_miss path=1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test error="error: path '/nix/store/1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 62.732186] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 62.901989] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.34 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker2 # [ 63.068606] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43424 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 63.078666] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 63.129582] nix-grpc-daemon[765]: ts=2026-09-23T07:00:02Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 63.136238] nix-grpc-daemon[765]: ts=2026-09-23T07:00:02Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 63.176626] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 63.192456] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 client # copying 1 paths to 'lb:50051'... worker2 # [ 63.208104] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker2 # [ 63.229677] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 63.270919] nix-grpc-daemon[770]: ts=2026-09-23T07:00:02Z level=info event=niks3_push_started pid=854 worker2 # [ 63.457354] nix-grpc-daemon[854]: time=2026-09-23T07:00:03.089Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 63.570910] nix-daemon[842]: accepted connection from pid 867, user nix-grpc-daemon (trusted) worker2 # [ 63.591784] nix-daemon[842]: reaped child process 870, status = succeeded lb # [ 63.675560] niks3-server[1015]: 2026/09/23 07:00:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 63.671520] nix-grpc-daemon[854]: time=2026-09-23T07:00:03.301Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 63.677195] nix-grpc-daemon[854]: time=2026-09-23T07:00:03.302Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 63.779409] niks3-server[1015]: 2026/09/23 07:00:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 63.839723] niks3-server[1015]: 2026/09/23 07:00:03 INFO Signed narinfos id=1 count=1 lb # [ 63.844254] niks3-server[1015]: 2026/09/23 07:00:03 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst worker2 # [ 63.770817] nix-grpc-daemon[854]: time=2026-09-23T07:00:03.401Z level=INFO msg="Uploading 1 narinfos" lb # [ 63.856380] niks3-server[1015]: 2026/09/23 07:00:03 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 63.865458] niks3-server[1015]: 2026/09/23 07:00:03 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 63.873881] niks3-server[1015]: 2026/09/23 07:00:03 INFO Completed upload id=1 worker2 # [ 63.800809] nix-grpc-daemon[854]: time=2026-09-23T07:00:03.432Z level=INFO msg="Upload complete. (339ms)" lb # [ 63.877165] niks3-server[1015]: 2026/09/23 07:00:03 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo worker2 # [ 63.811613] nix-grpc-daemon[770]: ts=2026-09-23T07:00:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 63.828863] nix-grpc-daemon[770]: ts=2026-09-23T07:00:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 63.840948] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 63.848337] nix-daemon[842]: reaped child process 852, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker2 # [ 63.864195] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker2 # [ 64.931781] nix-daemon[842]: reaped child process 874, status = succeeded worker2 # [ 64.994643] nix-daemon[842]: accepted connection from pid 890, user nix-grpc-daemon (trusted) worker2 # [ 65.014545] nix-daemon[842]: reaped child process 893, status = succeeded lb # [ 65.095080] niks3-server[1015]: 2026/09/23 07:00:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 65.036473] nix-grpc-daemon[854]: time=2026-09-23T07:00:04.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 65.038230] nix-grpc-daemon[854]: time=2026-09-23T07:00:04.666Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 65.135473] niks3-server[1015]: 2026/09/23 07:00:04 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker2 # [ 65.091958] nix-grpc-daemon[854]: time=2026-09-23T07:00:04.723Z level=INFO msg="Uploading 1 narinfos" lb # [ 65.156124] niks3-server[1015]: 2026/09/23 07:00:04 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 65.169189] niks3-server[1015]: 2026/09/23 07:00:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 65.177870] niks3-server[1015]: 2026/09/23 07:00:04 INFO Signed narinfos id=2 count=1 lb # [ 65.191493] niks3-server[1015]: 2026/09/23 07:00:04 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 65.196159] niks3-server[1015]: 2026/09/23 07:00:04 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 65.199994] niks3-server[1015]: 2026/09/23 07:00:04 INFO Completed upload id=2 worker2 # [ 65.126683] nix-grpc-daemon[854]: time=2026-09-23T07:00:04.758Z level=INFO msg="Upload complete. (251ms)" lb # [ 65.202637] niks3-server[1015]: 2026/09/23 07:00:04 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 65.158832] nix-grpc-daemon[765]: ts=2026-09-23T07:00:04Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker2 # [ 65.141266] nix-grpc-daemon[770]: ts=2026-09-23T07:00:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker2 # [ 65.160655] nix-daemon[842]: reaped child process 872, status = succeeded worker2 # [ 65.196614] nix-grpc-daemon[770]: ts=2026-09-23T07:00:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 65.212738] nix-grpc-daemon[770]: ts=2026-09-23T07:00:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 65.228106] nix-grpc-daemon[854]: time=2026-09-23T07:00:04.860Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 65.239123] nix-grpc-daemon[770]: ts=2026-09-23T07:00:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker2 # [ 65.261753] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 65.425314] nix-daemon[842]: accepted connection from pid 897, user nix-grpc-daemon (trusted) worker2 # [ 65.450273] nix-daemon[842]: reaped child process 900, status = succeeded lb # [ 65.521981] niks3-server[1015]: 2026/09/23 07:00:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 65.460184] nix-grpc-daemon[854]: time=2026-09-23T07:00:05.092Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 65.462314] nix-grpc-daemon[854]: time=2026-09-23T07:00:05.092Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 65.557115] niks3-server[1015]: 2026/09/23 07:00:05 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls worker2 # [ 65.496273] nix-grpc-daemon[854]: time=2026-09-23T07:00:05.128Z level=INFO msg="Uploading 1 narinfos" lb # [ 65.572888] niks3-server[1015]: 2026/09/23 07:00:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 65.582470] niks3-server[1015]: 2026/09/23 07:00:05 INFO Signed narinfos id=3 count=1 lb # [ 65.591658] niks3-server[1015]: 2026/09/23 07:00:05 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst worker2 # [ 65.523088] nix-grpc-daemon[854]: time=2026-09-23T07:00:05.155Z level=INFO msg="Upload complete. (242ms)" lb # [ 65.596873] niks3-server[1015]: 2026/09/23 07:00:05 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 65.600749] niks3-server[1015]: 2026/09/23 07:00:05 INFO Completed upload id=3 lb # [ 65.603677] niks3-server[1015]: 2026/09/23 07:00:05 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker2 # [ 65.532418] nix-grpc-daemon[770]: ts=2026-09-23T07:00:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 65.548161] nix-grpc-daemon[770]: ts=2026-09-23T07:00:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 65.554551] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 65.559548] nix-daemon[842]: reaped child process 895, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker2 # [ 65.574405] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker1 # [ 65.164831] nix-grpc-daemon[765]: ts=2026-09-23T07:00:04Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=1 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker2 # [ 66.836877] nix-daemon[842]: reaped child process 904, status = succeeded worker2 # [ 66.975966] nix-daemon[842]: accepted connection from pid 920, user nix-grpc-daemon (trusted) worker2 # [ 66.994016] nix-daemon[842]: reaped child process 923, status = succeeded lb # [ 67.075298] niks3-server[1015]: 2026/09/23 07:00:06 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 67.012437] nix-grpc-daemon[854]: time=2026-09-23T07:00:06.643Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 67.014218] nix-grpc-daemon[854]: time=2026-09-23T07:00:06.643Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 67.108198] niks3-server[1015]: 2026/09/23 07:00:06 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 67.123137] niks3-server[1015]: 2026/09/23 07:00:06 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst worker2 # [ 67.062194] nix-grpc-daemon[854]: time=2026-09-23T07:00:06.693Z level=INFO msg="Uploading 1 narinfos" lb # [ 67.137868] niks3-server[1015]: 2026/09/23 07:00:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 67.145881] niks3-server[1015]: 2026/09/23 07:00:06 INFO Signed narinfos id=4 count=1 lb # [ 67.150509] niks3-server[1015]: 2026/09/23 07:00:06 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 67.159885] niks3-server[1015]: 2026/09/23 07:00:06 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete worker2 # [ 67.086346] nix-grpc-daemon[854]: time=2026-09-23T07:00:06.718Z level=INFO msg="Upload complete. (299ms)" lb # [ 67.163453] niks3-server[1015]: 2026/09/23 07:00:06 INFO Completed upload id=4 lb # [ 67.165320] niks3-server[1015]: 2026/09/23 07:00:06 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 67.117314] nix-grpc-daemon[765]: ts=2026-09-23T07:00:06Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 67.117708] nix-grpc-daemon[765]: ts=2026-09-23T07:00:06Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 67.101169] nix-grpc-daemon[770]: ts=2026-09-23T07:00:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker2 # [ 67.119319] nix-daemon[842]: reaped child process 902, status = succeeded worker2 # [ 67.151081] nix-grpc-daemon[770]: ts=2026-09-23T07:00:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 67.171971] nix-grpc-daemon[770]: ts=2026-09-23T07:00:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 67.187678] nix-grpc-daemon[854]: time=2026-09-23T07:00:06.819Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 67.198017] nix-grpc-daemon[770]: ts=2026-09-23T07:00:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker2 # [ 67.220307] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 67.383831] nix-daemon[842]: accepted connection from pid 927, user nix-grpc-daemon (trusted) worker2 # [ 67.402933] nix-daemon[842]: reaped child process 930, status = succeeded lb # [ 67.480562] niks3-server[1015]: 2026/09/23 07:00:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 67.416142] nix-grpc-daemon[854]: time=2026-09-23T07:00:07.048Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 67.418320] nix-grpc-daemon[854]: time=2026-09-23T07:00:07.050Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" worker2 # [ 67.449653] nix-grpc-daemon[854]: time=2026-09-23T07:00:07.081Z level=INFO msg="Uploading 1 narinfos" lb # [ 67.512340] niks3-server[1015]: 2026/09/23 07:00:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 67.529965] niks3-server[1015]: 2026/09/23 07:00:07 INFO Signed narinfos id=5 count=1 lb # [ 67.536143] niks3-server[1015]: 2026/09/23 07:00:07 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 67.552522] niks3-server[1015]: 2026/09/23 07:00:07 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst worker2 # [ 67.487506] nix-grpc-daemon[854]: time=2026-09-23T07:00:07.119Z level=INFO msg="Upload complete. (249ms)" lb # [ 67.564155] niks3-server[1015]: 2026/09/23 07:00:07 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 67.570587] niks3-server[1015]: 2026/09/23 07:00:07 INFO Completed upload id=5 lb # [ 67.575335] niks3-server[1015]: 2026/09/23 07:00:07 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker2 # [ 67.500329] nix-grpc-daemon[770]: ts=2026-09-23T07:00:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 67.516393] nix-grpc-daemon[770]: ts=2026-09-23T07:00:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 67.526105] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 67.531580] nix-daemon[842]: reaped child process 925, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 67.548287] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) lb # [ 68.264891] envoy[895]: conn peer=[2001:db8:1::1]:47610 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=2284 lb # [ 68.269926] envoy[895]: conn peer=[2001:db8:1::1]:34242 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=361 ms=76 lb # [ 68.275322] envoy[895]: conn peer=[2001:db8:1::1]:34254 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=12 lb # [ 68.280059] envoy[895]: conn peer=[2001:db8:1::1]:34264 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=12 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker2 # [ 68.616775] nix-daemon[842]: reaped child process 934, status = succeeded worker2 # [ 68.727029] nix-daemon[842]: accepted connection from pid 950, user nix-grpc-daemon (trusted) worker2 # [ 68.744732] nix-daemon[842]: reaped child process 953, status = succeeded lb # [ 68.823577] niks3-server[1015]: 2026/09/23 07:00:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 68.763429] nix-grpc-daemon[854]: time=2026-09-23T07:00:08.393Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 68.765394] nix-grpc-daemon[854]: time=2026-09-23T07:00:08.393Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 68.870409] niks3-server[1015]: 2026/09/23 07:00:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker2 # [ 68.804524] nix-grpc-daemon[854]: time=2026-09-23T07:00:08.436Z level=INFO msg="Uploading 1 narinfos" lb # [ 68.888347] niks3-server[1015]: 2026/09/23 07:00:08 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 68.900650] niks3-server[1015]: 2026/09/23 07:00:08 INFO Signed narinfos id=6 count=1 lb # [ 68.907428] niks3-server[1015]: 2026/09/23 07:00:08 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 68.919966] niks3-server[1015]: 2026/09/23 07:00:08 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete worker2 # [ 68.845636] nix-grpc-daemon[854]: time=2026-09-23T07:00:08.477Z level=INFO msg="Upload complete. (285ms)" lb # [ 68.923341] niks3-server[1015]: 2026/09/23 07:00:08 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 68.928688] niks3-server[1015]: 2026/09/23 07:00:08 INFO Completed upload id=6 lb # [ 68.932495] niks3-server[1015]: 2026/09/23 07:00:08 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker2 # [ 68.857163] nix-grpc-daemon[770]: ts=2026-09-23T07:00:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker2 # [ 68.876905] nix-daemon[842]: reaped child process 932, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.98 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.18 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.21 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.49 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker1 # [ 70.097225] nix-grpc-daemon[765]: ts=2026-09-23T07:00:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49186 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 70.106949] nix-grpc-daemon[765]: ts=2026-09-23T07:00:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 70.140350] nix-grpc-daemon[765]: ts=2026-09-23T07:00:09Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.29 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.18 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 70.971411] nix-grpc-daemon[770]: ts=2026-09-23T07:00:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 70.979287] nix-grpc-daemon[770]: ts=2026-09-23T07:00:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 70.988134] nix-grpc-daemon[854]: time=2026-09-23T07:00:10.617Z level=INFO msg="All 2 paths already cached" worker1 # [ 71.009533] nix-grpc-daemon[765]: ts=2026-09-23T07:00:10Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 71.010117] nix-grpc-daemon[765]: ts=2026-09-23T07:00:10Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 71.043056] nix-grpc-daemon[770]: ts=2026-09-23T07:00:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 71.059599] nix-grpc-daemon[770]: ts=2026-09-23T07:00:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 71.074226] nix-grpc-daemon[854]: time=2026-09-23T07:00:10.706Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker2 # [ 71.085432] nix-grpc-daemon[770]: ts=2026-09-23T07:00:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 71.100571] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 71.252460] nix-daemon[842]: accepted connection from pid 970, user nix-grpc-daemon (trusted) worker2 # [ 71.274473] nix-daemon[842]: reaped child process 973, status = succeeded lb # [ 71.354437] niks3-server[1015]: 2026/09/23 07:00:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 71.291783] nix-grpc-daemon[854]: time=2026-09-23T07:00:10.923Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 71.294710] nix-grpc-daemon[854]: time=2026-09-23T07:00:10.923Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" worker2 # [ 71.322240] nix-grpc-daemon[854]: time=2026-09-23T07:00:10.954Z level=INFO msg="Uploading 1 narinfos" lb # [ 71.386772] niks3-server[1015]: 2026/09/23 07:00:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 71.402815] niks3-server[1015]: 2026/09/23 07:00:11 INFO Signed narinfos id=7 count=1 lb # [ 71.408888] niks3-server[1015]: 2026/09/23 07:00:11 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst lb # [ 71.419711] niks3-server[1015]: 2026/09/23 07:00:11 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 71.426745] niks3-server[1015]: 2026/09/23 07:00:11 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 71.433548] niks3-server[1015]: 2026/09/23 07:00:11 INFO Completed upload id=7 worker2 # [ 71.360416] nix-grpc-daemon[854]: time=2026-09-23T07:00:10.992Z level=INFO msg="Upload complete. (242ms)" lb # [ 71.439503] niks3-server[1015]: 2026/09/23 07:00:11 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker2 # [ 71.372315] nix-grpc-daemon[770]: ts=2026-09-23T07:00:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 71.388440] nix-grpc-daemon[770]: ts=2026-09-23T07:00:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 71.401298] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 71.405046] nix-daemon[842]: reaped child process 968, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker2 # [ 71.417554] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker2 # [ 72.485750] nix-daemon[842]: reaped child process 977, status = succeeded worker2 # [ 72.603651] nix-daemon[842]: accepted connection from pid 993, user nix-grpc-daemon (trusted) worker2 # [ 72.619549] nix-daemon[842]: reaped child process 996, status = succeeded lb # [ 72.700901] niks3-server[1015]: 2026/09/23 07:00:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 72.640751] nix-grpc-daemon[854]: time=2026-09-23T07:00:12.269Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 72.642468] nix-grpc-daemon[854]: time=2026-09-23T07:00:12.270Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 72.744641] niks3-server[1015]: 2026/09/23 07:00:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker2 # [ 72.681842] nix-grpc-daemon[854]: time=2026-09-23T07:00:12.313Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.762317] niks3-server[1015]: 2026/09/23 07:00:12 INFO Signed narinfos id=8 count=1 lb # [ 72.768962] niks3-server[1015]: 2026/09/23 07:00:12 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 72.776933] niks3-server[1015]: 2026/09/23 07:00:12 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 72.780983] niks3-server[1015]: 2026/09/23 07:00:12 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 72.715808] nix-grpc-daemon[854]: time=2026-09-23T07:00:12.347Z level=INFO msg="Upload complete. (285ms)" lb # [ 72.789751] niks3-server[1015]: 2026/09/23 07:00:12 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 72.793738] niks3-server[1015]: 2026/09/23 07:00:12 INFO Completed upload id=8 lb # [ 72.795526] niks3-server[1015]: 2026/09/23 07:00:12 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker2 # [ 72.729312] nix-grpc-daemon[770]: ts=2026-09-23T07:00:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker2 # [ 72.747235] nix-daemon[842]: reaped child process 975, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.01 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.52 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker2 # [ 73.325734] nix-grpc-daemon[770]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 73.342598] nix-grpc-daemon[765]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 73.366209] nix-grpc-daemon[765]: ts=2026-09-23T07:00:12Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker1 # [ 73.376663] nix-grpc-daemon[765]: ts=2026-09-23T07:00:12Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 73.385079] nix-grpc-daemon[765]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 73.392934] nix-grpc-daemon[765]: ts=2026-09-23T07:00:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker1 # [ 73.404636] nix-grpc-daemon[765]: ts=2026-09-23T07:00:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 73.421674] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 73.446771] nix-grpc-daemon[765]: ts=2026-09-23T07:00:13Z level=info event=niks3_push_started pid=902 worker1 # [ 73.605450] nix-grpc-daemon[902]: time=2026-09-23T07:00:13.228Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 73.711933] nix-daemon[852]: accepted connection from pid 915, user nix-grpc-daemon (trusted) worker1 # [ 73.731637] nix-daemon[852]: reaped child process 918, status = succeeded lb # [ 73.794967] niks3-server[1015]: 2026/09/23 07:00:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 73.743481] nix-grpc-daemon[902]: time=2026-09-23T07:00:13.366Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 73.747277] nix-grpc-daemon[902]: time=2026-09-23T07:00:13.366Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 73.833686] niks3-server[1015]: 2026/09/23 07:00:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign worker1 # [ 73.784070] nix-grpc-daemon[902]: time=2026-09-23T07:00:13.407Z level=INFO msg="Uploading 1 narinfos" lb # [ 73.849648] niks3-server[1015]: 2026/09/23 07:00:13 INFO Signed narinfos id=9 count=1 lb # [ 73.855668] niks3-server[1015]: 2026/09/23 07:00:13 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls lb # [ 73.863520] niks3-server[1015]: 2026/09/23 07:00:13 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst lb # [ 73.870517] niks3-server[1015]: 2026/09/23 07:00:13 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete worker1 # [ 73.816062] nix-grpc-daemon[902]: time=2026-09-23T07:00:13.439Z level=INFO msg="Upload complete. (206ms)" lb # [ 73.878269] niks3-server[1015]: 2026/09/23 07:00:13 INFO Completed upload id=9 lb # [ 73.881111] niks3-server[1015]: 2026/09/23 07:00:13 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker1 # [ 73.822789] nix-grpc-daemon[765]: ts=2026-09-23T07:00:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 73.834656] nix-grpc-daemon[765]: ts=2026-09-23T07:00:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 73.842340] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 73.844510] nix-daemon[852]: reaped child process 900, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... worker1 # [ 73.855726] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker1 # [ 74.724386] nix-daemon[852]: reaped child process 922, status = succeeded worker1 # [ 74.841757] nix-daemon[852]: accepted connection from pid 938, user nix-grpc-daemon (trusted) worker1 # [ 74.859966] nix-daemon[852]: reaped child process 941, status = succeeded lb # [ 74.920923] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 74.872537] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.493Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 74.874054] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.493Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 74.960385] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign worker1 # [ 74.908883] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.532Z level=INFO msg="Uploading 1 narinfos" lb # [ 74.975440] niks3-server[1015]: 2026/09/23 07:00:14 INFO Signed narinfos id=10 count=1 lb # [ 74.986274] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst worker1 # [ 74.943391] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.567Z level=INFO msg="Upload complete. (278ms)" lb # [ 75.004826] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 75.014777] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 75.019825] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete worker1 # [ 74.959675] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 lb # [ 75.027145] niks3-server[1015]: 2026/09/23 07:00:14 INFO Completed upload id=10 lb # [ 75.030192] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker1 # [ 74.976711] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker1 # [ 74.982581] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 74.987327] nix-daemon[852]: reaped child process 920, status = succeeded worker1 # [ 75.014823] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 75.029871] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 75.043324] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.667Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker1 # [ 75.053883] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 75.070318] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 75.212913] nix-daemon[852]: accepted connection from pid 947, user nix-grpc-daemon (trusted) worker1 # [ 75.232453] nix-daemon[852]: reaped child process 950, status = succeeded lb # [ 75.297928] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 75.245410] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.868Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 75.247106] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.868Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 75.328090] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign worker1 # [ 75.275023] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.898Z level=INFO msg="Uploading 1 narinfos" lb # [ 75.347511] niks3-server[1015]: 2026/09/23 07:00:14 INFO Signed narinfos id=11 count=1 lb # [ 75.357354] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst lb # [ 75.373490] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls worker1 # [ 75.318724] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.942Z level=INFO msg="Upload complete. (236ms)" lb # [ 75.381563] niks3-server[1015]: 2026/09/23 07:00:15 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 75.385368] niks3-server[1015]: 2026/09/23 07:00:15 INFO Completed upload id=11 lb # [ 75.388239] niks3-server[1015]: 2026/09/23 07:00:15 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker1 # [ 75.328342] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 75.348569] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 75.353420] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 75.364252] nix-daemon[852]: reaped child process 945, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... worker1 # [ 75.372154] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker1 # [ 76.198045] nix-daemon[852]: reaped child process 954, status = succeeded worker1 # [ 76.308474] nix-daemon[852]: accepted connection from pid 970, user nix-grpc-daemon (trusted) worker1 # [ 76.324390] nix-daemon[852]: reaped child process 973, status = succeeded lb # [ 76.394219] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 76.340932] nix-grpc-daemon[902]: time=2026-09-23T07:00:15.963Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 76.342360] nix-grpc-daemon[902]: time=2026-09-23T07:00:15.963Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" worker1 # [ 76.376419] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.000Z level=INFO msg="Uploading 1 narinfos" lb # [ 76.428797] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 76.442272] niks3-server[1015]: 2026/09/23 07:00:16 INFO Signed narinfos id=12 count=1 lb # [ 76.453637] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst worker1 # [ 76.406815] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.030Z level=INFO msg="Upload complete. (225ms)" lb # [ 76.470152] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls lb # [ 76.477936] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 76.485025] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete worker1 # [ 76.421653] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 lb # [ 76.489855] niks3-server[1015]: 2026/09/23 07:00:16 INFO Completed upload id=12 lb # [ 76.492485] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker1 # [ 76.437027] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker1 # [ 76.442344] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 76.446901] nix-daemon[852]: reaped child process 952, status = succeeded worker1 # [ 76.482766] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 76.497258] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 76.510618] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.134Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker1 # [ 76.522490] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 76.539208] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 76.685289] nix-daemon[852]: accepted connection from pid 979, user nix-grpc-daemon (trusted) worker1 # [ 76.702191] nix-daemon[852]: reaped child process 982, status = succeeded lb # [ 76.770394] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 76.717165] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.341Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 76.719247] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.342Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 76.800154] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls worker1 # [ 76.747456] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.370Z level=INFO msg="Uploading 1 narinfos" lb # [ 76.811650] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 76.823430] niks3-server[1015]: 2026/09/23 07:00:16 INFO Signed narinfos id=13 count=1 lb # [ 76.833888] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst worker1 # [ 76.778754] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.402Z level=INFO msg="Upload complete. (226ms)" lb # [ 76.845228] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 76.850501] niks3-server[1015]: 2026/09/23 07:00:16 INFO Completed upload id=13 worker1 # [ 76.788270] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 76.853471] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker1 # [ 76.798874] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 76.808399] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 76.811757] nix-daemon[852]: reaped child process 977, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... worker1 # [ 76.822434] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig lb # [ 78.264275] envoy[895]: conn peer=[2001:db8:1::1]:34270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9562 tx=4389 ms=5808 lb # [ 78.264604] envoy[895]: conn peer=[2001:db8:1::1]:34282 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=66 lb # [ 78.264726] envoy[895]: conn peer=[2001:db8:1::1]:52174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1893 ms=1776 worker1 # [ 78.289445] nix-daemon[852]: reaped child process 986, status = succeeded worker1 # [ 78.392851] nix-daemon[852]: accepted connection from pid 1002, user nix-grpc-daemon (trusted) worker1 # [ 78.410663] nix-daemon[852]: reaped child process 1005, status = succeeded lb # [ 78.479069] niks3-server[1015]: 2026/09/23 07:00:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 78.427838] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.049Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 78.429525] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.049Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 78.512571] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls lb # [ 78.526613] niks3-server[1015]: 2026/09/23 07:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker1 # [ 78.471577] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.095Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.540531] niks3-server[1015]: 2026/09/23 07:00:18 INFO Signed narinfos id=14 count=1 lb # [ 78.546492] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 78.556434] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst worker1 # [ 78.501023] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.124Z level=INFO msg="Upload complete. (266ms)" lb # [ 78.563061] niks3-server[1015]: 2026/09/23 07:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 78.567235] niks3-server[1015]: 2026/09/23 07:00:18 INFO Completed upload id=14 lb # [ 78.572846] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker1 # [ 78.517055] nix-grpc-daemon[765]: ts=2026-09-23T07:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker1 # [ 78.530839] nix-daemon[852]: reaped child process 984, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.43 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.18 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 78.862384] nix-grpc-daemon[765]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:33360 worker1 # [ 78.870619] nix-grpc-daemon[765]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:33360 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.20 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 79.063498] nix-grpc-daemon[770]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:33874 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 79.110284] nix-grpc-daemon[770]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:33874 worker2 # [ 79.119247] nix-grpc-daemon[770]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker2 # [ 79.135625] nix-grpc-daemon[770]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker2 # [ 79.151402] nix-grpc-daemon[770]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.28 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/837" client # removing stale temporary roots file "/nix/var/nix/temproots/804" client # removing stale temporary roots file "/nix/var/nix/temproots/866" client # removing stale temporary roots file "/nix/var/nix/temproots/881" client # removing stale temporary roots file "/nix/var/nix/temproots/858" client # removing stale temporary roots file "/nix/var/nix/temproots/825" client # removing stale temporary roots file "/nix/var/nix/temproots/845" client # removing stale temporary roots file "/nix/var/nix/temproots/789" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker1 # [ 79.541542] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49186 worker2 # [ 79.532179] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 79.543481] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.174Z level=INFO msg="All 1 paths already cached" worker2 # [ 79.551334] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 79.572263] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:49186 client # copying 3 paths... worker2 # [ 79.573648] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 79.599930] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 79.607296] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=232 worker2 # [ 79.616320] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 79.621300] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 79.630062] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 79.635142] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.53 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 6.61 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 79.775333] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 213b01d5f2ab4ba384a40626201ef013 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup worker1 # [ 79.976467] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49186 client # [ 79.966074] nix[934]: don't know how to build these paths: client # [ 79.967848] nix[934]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 79.994433] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 80.000580] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 80.031898] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 80.044819] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 client # [ 80.052150] nix[934]: copying 1 paths to 'lb:50051'... worker2 # [ 80.058808] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 client # [ 80.056168] nix[934]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker2 # [ 80.075027] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 80.237872] nix-daemon[842]: accepted connection from pid 1003, user nix-grpc-daemon (trusted) worker2 # [ 80.259750] nix-daemon[842]: reaped child process 1006, status = succeeded lb # [ 80.332912] niks3-server[1015]: 2026/09/23 07:00:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 80.268951] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.900Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 80.270468] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.900Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 80.369089] niks3-server[1015]: 2026/09/23 07:00:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker2 # [ 80.304560] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.936Z level=INFO msg="Uploading 1 narinfos" lb # [ 80.379582] niks3-server[1015]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 80.392337] niks3-server[1015]: 2026/09/23 07:00:20 INFO Signed narinfos id=15 count=1 lb # [ 80.397462] niks3-server[1015]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 80.407157] niks3-server[1015]: 2026/09/23 07:00:20 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 80.412838] niks3-server[1015]: 2026/09/23 07:00:20 INFO Completed upload id=15 worker2 # [ 80.338487] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.970Z level=INFO msg="Upload complete. (245ms)" lb # [ 80.417686] niks3-server[1015]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker2 # [ 80.349264] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 80.367825] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 80.379714] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 80.384638] nix-daemon[842]: reaped child process 1001, status = succeeded client # [ 80.390259] nix[934]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker2 # [ 80.400162] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # [ 80.870667] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 76c0ac2a1cd5466f9c88faa9998a9010 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) client: must succeed: sleep 2 worker2 # [ 81.118894] nix-grpc-daemon[770]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43424 client # [ 81.121512] nix[950]: this derivation will be built: worker2 # [ 81.129381] nix-grpc-daemon[770]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 client # [ 81.127288] nix[950]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 81.155878] nix-grpc-daemon[765]: ts=2026-09-23T07:00:20Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 81.148111] nix-grpc-daemon[770]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 client # [ 81.147701] nix[950]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client: (finished: must succeed: sleep 2, in 2.05 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker2 # [ 83.484848] nix-daemon[842]: reaped child process 1010, status = succeeded worker2 # [ 83.611293] nix-daemon[842]: accepted connection from pid 1036, user nix-grpc-daemon (trusted) worker2 # [ 83.636097] nix-daemon[842]: reaped child process 1039, status = succeeded lb # [ 83.715749] niks3-server[1015]: 2026/09/23 07:00:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 83.656427] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.285Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 83.657958] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.285Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 83.777831] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 83.710912] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.342Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.790659] niks3-server[1015]: 2026/09/23 07:00:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 83.798157] niks3-server[1015]: 2026/09/23 07:00:23 INFO Signed narinfos id=16 count=1 lb # [ 83.805116] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 83.814501] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls worker2 # [ 83.740617] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.372Z level=INFO msg="Upload complete. (325ms)" lb # [ 83.818088] niks3-server[1015]: 2026/09/23 07:00:23 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 83.823394] niks3-server[1015]: 2026/09/23 07:00:23 INFO Completed upload id=16 lb # [ 83.826352] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker2 # [ 83.753283] nix-grpc-daemon[770]: ts=2026-09-23T07:00:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 83.773197] nix-grpc-daemon[770]: ts=2026-09-23T07:00:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=2 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 83.785161] nix-daemon[842]: reaped child process 1008, status = succeeded client # [ 83.816505] systemd[1]: dedup2.service: Deactivated successfully. client # [ 83.821777] systemd[1]: dedup2.service: Consumed 217ms CPU time over 2.941s wall clock time, 35.2M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic. client # [ 83.854125] systemd[1]: dedup1.service: Deactivated successfully. client # [ 83.857873] systemd[1]: dedup1.service: Consumed 173ms CPU time over 4.082s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 5.00 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 84.954265] nix-grpc-daemon[770]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 84.986528] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker1 # [ 84.986917] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 85.001990] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 85.013022] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 85.027147] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 85.042562] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 85.196958] nix-daemon[852]: accepted connection from pid 1032, user nix-grpc-daemon (trusted) worker1 # [ 85.217256] nix-daemon[852]: reaped child process 1035, status = succeeded lb # [ 85.285161] niks3-server[1015]: 2026/09/23 07:00:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 85.234724] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.857Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 85.236254] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.857Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 85.315837] niks3-server[1015]: 2026/09/23 07:00:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker1 # [ 85.263979] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.887Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.331523] niks3-server[1015]: 2026/09/23 07:00:24 INFO Signed narinfos id=17 count=1 lb # [ 85.338592] niks3-server[1015]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls worker1 # [ 85.289135] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.912Z level=INFO msg="Upload complete. (227ms)" lb # [ 85.350695] niks3-server[1015]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst lb # [ 85.360096] niks3-server[1015]: 2026/09/23 07:00:24 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 85.365236] niks3-server[1015]: 2026/09/23 07:00:24 INFO Completed upload id=17 lb # [ 85.367671] niks3-server[1015]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker1 # [ 85.296243] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 85.310415] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 85.316544] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 85.321405] nix-daemon[852]: reaped child process 1030, status = succeeded worker1 # [ 85.334237] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) lb # [ 88.265383] envoy[895]: conn peer=[2001:db8:1::1]:52178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9485 tx=4349 ms=5189 lb # [ 88.270289] envoy[895]: conn peer=[2001:db8:1::1]:52186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=178 lb # [ 88.275225] envoy[895]: conn peer=[2001:db8:1::1]:52196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=107 lb # [ 88.280079] envoy[895]: conn peer=[2001:db8:1::1]:52210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=79 lb # [ 88.285635] envoy[895]: conn peer=[2001:db8:1::1]:52198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=94 lb # [ 88.290803] envoy[895]: conn peer=[2001:db8:1::1]:38274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1597 ms=2655 lb # [ 88.296503] envoy[895]: conn peer=[2001:db8:1::1]:52226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1564 ms=3851 worker1 # [ 91.548505] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker1 # [ 91.571212] nix-daemon[852]: reaped child process 1037, status = succeeded worker1 # [ 91.573004] nix-daemon[852]: reaped child process 1039, status = succeeded worker1 # [ 91.791073] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 91.807596] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker2 # [ 91.801264] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 91.814123] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 91.817945] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 91.832672] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 91.847708] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 92.014803] nix-daemon[842]: accepted connection from pid 1052, user nix-grpc-daemon (trusted) worker2 # [ 92.034376] nix-daemon[842]: reaped child process 1055, status = succeeded lb # [ 92.113987] niks3-server[1015]: 2026/09/23 07:00:31 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 92.051613] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.682Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 92.053721] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.682Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 92.149384] niks3-server[1015]: 2026/09/23 07:00:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker2 # [ 92.086915] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.718Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.163870] niks3-server[1015]: 2026/09/23 07:00:31 INFO Signed narinfos id=18 count=1 lb # [ 92.172335] niks3-server[1015]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 92.181320] niks3-server[1015]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 92.195906] niks3-server[1015]: 2026/09/23 07:00:31 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 92.202834] niks3-server[1015]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker2 # [ 92.129380] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.761Z level=INFO msg="Upload complete. (262ms)" lb # [ 92.206783] niks3-server[1015]: 2026/09/23 07:00:31 INFO Completed upload id=18 worker2 # [ 92.139678] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 92.153697] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 92.165693] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 92.172423] nix-daemon[842]: reaped child process 1050, status = succeeded worker2 # [ 92.184940] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 98.002011] nix-grpc-daemon[770]: ts=2026-09-23T07:00:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker2 # [ 98.042026] nix-daemon[842]: reaped child process 1057, status = succeeded worker2 # [ 98.047412] nix-daemon[842]: reaped child process 1059, status = succeeded lb # [ 98.264755] envoy[895]: conn peer=[2001:db8:1::1]:38276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2984 tx=957 ms=6606 (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.51 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 98.296982] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: f219f3c637ce498399b0ba4af614ee54 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share worker2 # [ 98.501816] nix-grpc-daemon[770]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43424 client # [ 98.503401] nix[1022]: don't know how to build these paths: client # [ 98.504538] nix[1022]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 98.536754] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 98.537050] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 98.549411] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 98.565736] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 client # [ 98.559800] nix[1022]: copying 1 paths to 'lb:50051'... client # [ 98.563925] nix[1022]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker1 # [ 98.581547] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 98.596247] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 98.750352] nix-daemon[852]: accepted connection from pid 1066, user nix-grpc-daemon (trusted) worker1 # [ 98.769215] nix-daemon[852]: reaped child process 1069, status = succeeded lb # [ 98.270115] envoy[895]: conn peer=[2001:db8:1::1]:40694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3002 tx=953 ms=6283 lb # [ 98.836785] niks3-server[1015]: 2026/09/23 07:00:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 98.789979] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.413Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 98.791639] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.413Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 98.875496] niks3-server[1015]: 2026/09/23 07:00:38 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst worker1 # [ 98.829363] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.452Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.890716] niks3-server[1015]: 2026/09/23 07:00:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign lb # [ 98.901779] niks3-server[1015]: 2026/09/23 07:00:38 INFO Signed narinfos id=19 count=1 lb # [ 98.907579] niks3-server[1015]: 2026/09/23 07:00:38 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 98.922543] niks3-server[1015]: 2026/09/23 07:00:38 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 98.928488] niks3-server[1015]: 2026/09/23 07:00:38 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker1 # [ 98.866573] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.490Z level=INFO msg="Upload complete. (254ms)" lb # [ 98.932188] niks3-server[1015]: 2026/09/23 07:00:38 INFO Completed upload id=19 worker1 # [ 98.876987] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 98.892971] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 98.907832] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 98.911684] nix-daemon[852]: reaped child process 1064, status = succeeded worker1 # [ 98.924422] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # [ 98.905602] nix[1022]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 99.402744] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 7bb2a0380c28405a9f2053d2b1716a6b client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds) worker2 # [ 99.598239] nix-grpc-daemon[770]: ts=2026-09-23T07:00:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43424 client # [ 99.599790] nix[1038]: don't know how to build these paths: client # [ 99.601532] nix[1038]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 99.630065] nix-grpc-daemon[765]: ts=2026-09-23T07:00:39Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 client # [ 99.620505] nix[1038]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker1 # [ 99.637461] nix-grpc-daemon[765]: ts=2026-09-23T07:00:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 102.777225] systemd[1]: share1.service: Sent signal SIGINT to main process 1022 (nix) on client request. client # [ 102.786497] nix[1022]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 102.800539] nix[1022]: error: interrupted by the user client # [ 102.864317] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 102.866264] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 102.869643] systemd[1]: share1.service: Consumed 170ms CPU time over 4.572s wall clock time, 43.5M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds) worker1 # [ 104.158021] nix-daemon[852]: reaped child process 1073, status = succeeded client: waiting for success: ! systemctl is-active share2 worker1 # [ 104.245840] nix-daemon[852]: accepted connection from pid 1109, user nix-grpc-daemon (trusted) worker1 # [ 104.262228] nix-daemon[852]: reaped child process 1112, status = succeeded lb # [ 104.332733] niks3-server[1015]: 2026/09/23 07:00:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 104.284367] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 104.285830] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.907Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 104.379663] niks3-server[1015]: 2026/09/23 07:00:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign worker1 # [ 104.327523] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.951Z level=INFO msg="Uploading 1 narinfos" lb # [ 104.393304] niks3-server[1015]: 2026/09/23 07:00:44 INFO Signed narinfos id=20 count=1 lb # [ 104.402837] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 104.411080] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 104.423085] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 104.436468] niks3-server[1015]: 2026/09/23 07:00:44 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete worker1 # [ 104.374537] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.998Z level=INFO msg="Upload complete. (234ms)" lb # [ 104.445040] niks3-server[1015]: 2026/09/23 07:00:44 INFO Completed upload id=20 lb # [ 104.449318] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker1 # [ 104.392247] nix-grpc-daemon[765]: ts=2026-09-23T07:00:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker1 # [ 104.405772] nix-grpc-daemon[765]: ts=2026-09-23T07:00:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker1 # [ 104.412697] nix-daemon[852]: reaped child process 1071, status = succeeded client # [ 104.404620] systemd[1]: share2.service: Deactivated successfully. client # [ 104.409617] systemd[1]: share2.service: Consumed 173ms CPU time over 5.003s wall clock time, 33.3M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds) (finished: subtest: the first client leaving does not fail the second, in 7.11 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 105.475383] systemd[1]: Started Nix Daemon. client # [ 105.592622] nix-daemon[1090]: accepted connection from pid 1087, user root (trusted) worker1 # [ 105.752681] nix-grpc-daemon[765]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:49186 worker2 # [ 105.745671] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 105.768277] nix-grpc-daemon[765]: ts=2026-09-23T07:00:45Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 105.774354] nix-grpc-daemon[765]: ts=2026-09-23T07:00:45Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 105.765093] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 105.780304] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 105.793787] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 105.808894] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 105.962747] nix-daemon[842]: accepted connection from pid 1098, user nix-grpc-daemon (trusted) worker2 # [ 105.981824] nix-daemon[842]: reaped child process 1101, status = succeeded lb # [ 106.061159] niks3-server[1015]: 2026/09/23 07:00:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 105.997889] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.629Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 105.999515] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.629Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 106.090051] niks3-server[1015]: 2026/09/23 07:00:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker2 # [ 106.025717] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.657Z level=INFO msg="Uploading 1 narinfos" lb # [ 106.104721] niks3-server[1015]: 2026/09/23 07:00:45 INFO Signed narinfos id=21 count=1 lb # [ 106.113128] niks3-server[1015]: 2026/09/23 07:00:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 106.121460] niks3-server[1015]: 2026/09/23 07:00:45 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 106.132733] niks3-server[1015]: 2026/09/23 07:00:45 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete worker2 # [ 106.064308] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.696Z level=INFO msg="Upload complete. (238ms)" lb # [ 106.139306] niks3-server[1015]: 2026/09/23 07:00:45 INFO Completed upload id=21 lb # [ 106.142364] niks3-server[1015]: 2026/09/23 07:00:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker2 # [ 106.076858] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 106.088815] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 106.097749] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 106.107488] nix-daemon[842]: reaped child process 1096, status = succeeded worker2 # [ 106.114057] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 107.376788] nix-daemon[842]: reaped child process 1105, status = succeeded worker2 # [ 107.470879] nix-daemon[842]: accepted connection from pid 1121, user nix-grpc-daemon (trusted) worker2 # [ 107.489006] nix-daemon[842]: reaped child process 1124, status = succeeded lb # [ 107.568893] niks3-server[1015]: 2026/09/23 07:00:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 107.505146] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.136Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 107.506669] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.136Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 107.607548] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 107.624402] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker2 # [ 107.559376] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.191Z level=INFO msg="Uploading 1 narinfos" lb # [ 107.636456] niks3-server[1015]: 2026/09/23 07:00:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 107.639874] niks3-server[1015]: 2026/09/23 07:00:47 INFO Signed narinfos id=22 count=1 lb # [ 107.647999] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 107.656164] niks3-server[1015]: 2026/09/23 07:00:47 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete worker2 # [ 107.582677] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.214Z level=INFO msg="Upload complete. (257ms)" lb # [ 107.659795] niks3-server[1015]: 2026/09/23 07:00:47 INFO Completed upload id=22 lb # [ 107.662164] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker2 # [ 107.596464] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker2 # [ 107.611055] nix-daemon[842]: reaped child process 1103, status = succeeded worker2 # [ 107.642438] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 107.690424] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 107.704590] nix-grpc-daemon[765]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker1 # [ 107.722354] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 108.003637] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:49186 worker2 # [ 107.995949] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 108.005380] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.637Z level=INFO msg="All 1 paths already cached" worker1 # [ 108.033357] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 108.038999] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 108.067235] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 108.082618] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 108.097146] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.729Z level=INFO msg="All 1 paths already cached" worker2 # [ 108.107036] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 108.121005] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) lb # [ 108.268792] envoy[895]: conn peer=[2001:db8:1::1]:40696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3029 tx=787 ms=4310 lb # [ 108.269127] envoy[895]: conn peer=[2001:db8:1::1]:40700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=4795 lb # [ 108.269248] envoy[895]: conn peer=[2001:db8:1::1]:48186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3410 tx=1926 ms=2027 lb # [ 108.269347] envoy[895]: conn peer=[2001:db8:1::1]:48196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=100 worker2 # [ 108.280708] nix-daemon[842]: accepted connection from pid 1128, user nix-grpc-daemon (trusted) worker2 # [ 108.302295] nix-daemon[842]: reaped child process 1131, status = succeeded lb # [ 108.373546] niks3-server[1015]: 2026/09/23 07:00:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 108.309124] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.940Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 108.310563] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.941Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" worker2 # [ 108.340815] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.972Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.405106] niks3-server[1015]: 2026/09/23 07:00:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 108.423841] niks3-server[1015]: 2026/09/23 07:00:48 INFO Signed narinfos id=23 count=1 lb # [ 108.431652] niks3-server[1015]: 2026/09/23 07:00:48 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 108.442620] niks3-server[1015]: 2026/09/23 07:00:48 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker2 # [ 108.379262] nix-grpc-daemon[854]: time=2026-09-23T07:00:48.011Z level=INFO msg="Upload complete. (241ms)" lb # [ 108.453946] niks3-server[1015]: 2026/09/23 07:00:48 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 108.462282] niks3-server[1015]: 2026/09/23 07:00:48 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo lb # [ 108.467272] niks3-server[1015]: 2026/09/23 07:00:48 INFO Completed upload id=23 worker2 # [ 108.388294] nix-grpc-daemon[770]: ts=2026-09-23T07:00:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 108.401191] nix-grpc-daemon[770]: ts=2026-09-23T07:00:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 108.415816] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 108.424203] nix-daemon[842]: reaped child process 1126, status = succeeded worker2 # [ 108.433828] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 110.139520] nix-daemon[842]: reaped child process 1135, status = succeeded worker2 # [ 110.214401] nix-daemon[842]: accepted connection from pid 1151, user nix-grpc-daemon (trusted) worker2 # [ 110.232963] nix-daemon[842]: reaped child process 1154, status = succeeded lb # [ 110.313007] niks3-server[1015]: 2026/09/23 07:00:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 110.251944] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.880Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 110.253860] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.880Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 110.352471] niks3-server[1015]: 2026/09/23 07:00:49 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 110.366607] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign worker2 # [ 110.298279] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.930Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.376300] niks3-server[1015]: 2026/09/23 07:00:50 INFO Signed narinfos id=24 count=1 lb # [ 110.384090] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls worker2 # [ 110.318681] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.950Z level=INFO msg="Upload complete. (273ms)" lb # [ 110.393166] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 110.400855] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 110.405038] niks3-server[1015]: 2026/09/23 07:00:50 INFO Completed upload id=24 worker2 # [ 110.329143] nix-grpc-daemon[770]: ts=2026-09-23T07:00:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 lb # [ 110.407255] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker2 # [ 110.347859] nix-grpc-daemon[770]: ts=2026-09-23T07:00:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 110.353346] nix-daemon[842]: reaped child process 1133, status = succeeded worker1 # [ 110.389246] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 110.402353] nix-grpc-daemon[765]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker1 # [ 110.419118] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 110.686380] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 110.701969] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 110.712390] nix-grpc-daemon[902]: time=2026-09-23T07:00:50.335Z level=INFO msg="All 2 paths already cached" worker1 # [ 110.729681] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker2 # [ 110.721353] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 110.736145] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 110.736649] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 110.752617] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.384Z level=INFO msg="All 2 paths already cached" worker2 # [ 110.762316] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 110.776128] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 110.934616] nix-daemon[842]: accepted connection from pid 1158, user nix-grpc-daemon (trusted) worker2 # [ 110.954279] nix-daemon[842]: reaped child process 1161, status = succeeded lb # [ 111.026728] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 110.962512] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.594Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 110.964262] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.594Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 111.056618] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker2 # [ 110.990762] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.622Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.071564] niks3-server[1015]: 2026/09/23 07:00:50 INFO Signed narinfos id=25 count=1 lb # [ 111.079748] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 111.085787] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls worker2 # [ 111.024036] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.656Z level=INFO msg="Upload complete. (230ms)" lb # [ 111.098817] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 111.103150] niks3-server[1015]: 2026/09/23 07:00:50 INFO Completed upload id=25 lb # [ 111.106304] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker2 # [ 111.032288] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 111.045175] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 111.056459] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 111.060506] nix-daemon[842]: reaped child process 1156, status = succeeded worker2 # [ 111.073830] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 111.707425] nix-daemon[842]: reaped child process 1165, status = succeeded worker2 # [ 111.813716] nix-daemon[842]: accepted connection from pid 1181, user nix-grpc-daemon (trusted) worker2 # [ 111.832179] nix-daemon[842]: reaped child process 1184, status = succeeded lb # [ 111.905707] niks3-server[1015]: 2026/09/23 07:00:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.842636] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.473Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 111.844181] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.473Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 111.947224] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker2 # [ 111.886834] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.518Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.963329] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 111.973768] niks3-server[1015]: 2026/09/23 07:00:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 111.980279] niks3-server[1015]: 2026/09/23 07:00:51 INFO Signed narinfos id=26 count=1 worker2 # [ 111.913772] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.545Z level=INFO msg="Upload complete. (230ms)" lb # [ 111.988010] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls lb # [ 111.991481] niks3-server[1015]: 2026/09/23 07:00:51 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 111.996552] niks3-server[1015]: 2026/09/23 07:00:51 INFO Completed upload id=26 lb # [ 111.998889] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker2 # [ 111.924982] nix-grpc-daemon[770]: ts=2026-09-23T07:00:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker2 # [ 111.941226] nix-grpc-daemon[770]: ts=2026-09-23T07:00:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 111.946767] nix-daemon[842]: reaped child process 1163, status = succeeded worker1 # [ 111.973055] nix-grpc-daemon[765]: ts=2026-09-23T07:00:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 111.985418] nix-grpc-daemon[765]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker1 # [ 112.002214] nix-grpc-daemon[765]: ts=2026-09-23T07:00:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=232 client # [ 112.084389] nix-daemon[1090]: reaped child process 1092, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.76 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.88 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 113.031207] nix-daemon[1090]: accepted connection from pid 1173, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 113.195595] nix-grpc-daemon[765]: ts=2026-09-23T07:00:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 113.203433] nix-grpc-daemon[765]: ts=2026-09-23T07:00:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 113.365573] nix-daemon[852]: accepted connection from pid 1128, user nix-grpc-daemon (trusted) worker1 # [ 113.383687] nix-daemon[852]: reaped child process 1131, status = succeeded lb # [ 113.454019] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 113.402722] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.025Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 113.404564] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.025Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 113.486666] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign worker1 # [ 113.435593] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.059Z level=INFO msg="Uploading 1 narinfos" lb # [ 113.500672] niks3-server[1015]: 2026/09/23 07:00:53 INFO Signed narinfos id=27 count=1 lb # [ 113.509583] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker1 # [ 113.463387] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.087Z level=INFO msg="Upload complete. (260ms)" lb # [ 113.525615] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 113.535585] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 113.541712] niks3-server[1015]: 2026/09/23 07:00:53 INFO Completed upload id=27 lb # [ 113.544693] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 113.490060] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 113.497316] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 113.506821] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 113.518387] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker1 # [ 113.536951] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker1 # [ 113.552491] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 113.704120] nix-daemon[852]: accepted connection from pid 1137, user nix-grpc-daemon (trusted) worker1 # [ 113.722513] nix-daemon[852]: reaped child process 1140, status = succeeded lb # [ 113.792179] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 113.802627] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 113.752100] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.373Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker1 # [ 113.755615] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.373Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker1 # [ 113.758336] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.373Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 113.841276] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 113.863488] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls worker1 # [ 113.813222] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.436Z level=INFO msg="Uploading 3 narinfos" lb # [ 113.875173] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 113.884816] niks3-server[1015]: 2026/09/23 07:00:53 INFO Signed narinfos id=28 count=1 lb # [ 113.889708] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 113.895681] niks3-server[1015]: 2026/09/23 07:00:53 INFO Signed narinfos id=29 count=2 lb # [ 113.899903] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 113.909343] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 113.917737] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 113.928512] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 113.931088] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 113.935913] niks3-server[1015]: 2026/09/23 07:00:53 INFO Completed upload id=28 lb # [ 113.938926] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 113.942878] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 113.945697] niks3-server[1015]: 2026/09/23 07:00:53 INFO Completed upload id=29 worker1 # [ 113.881185] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.504Z level=INFO msg="Upload complete. (311ms)" worker1 # [ 113.891922] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=2 nar_bytes_in=984 worker1 # [ 113.906938] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 113.921557] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 113.928575] nix-daemon[852]: reaped child process 1135, status = succeeded client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker1 # [ 113.947896] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 115.175305] nix-daemon[852]: reaped child process 1144, status = succeeded worker1 # [ 115.272111] nix-daemon[852]: accepted connection from pid 1160, user nix-grpc-daemon (trusted) worker1 # [ 115.286852] nix-daemon[852]: reaped child process 1163, status = succeeded lb # [ 115.359321] niks3-server[1015]: 2026/09/23 07:00:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 115.313605] nix-grpc-daemon[902]: time=2026-09-23T07:00:54.935Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 115.400802] niks3-server[1015]: 2026/09/23 07:00:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker1 # [ 115.348582] nix-grpc-daemon[902]: time=2026-09-23T07:00:54.972Z level=INFO msg="Uploading 1 narinfos" lb # [ 115.411987] niks3-server[1015]: 2026/09/23 07:00:55 INFO Signed narinfos id=30 count=1 lb # [ 115.418720] niks3-server[1015]: 2026/09/23 07:00:55 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 115.431087] niks3-server[1015]: 2026/09/23 07:00:55 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 115.441183] niks3-server[1015]: 2026/09/23 07:00:55 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete worker1 # [ 115.388543] nix-grpc-daemon[902]: time=2026-09-23T07:00:55.012Z level=INFO msg="Upload complete. (227ms)" lb # [ 115.450421] niks3-server[1015]: 2026/09/23 07:00:55 INFO Completed upload id=30 lb # [ 115.454681] niks3-server[1015]: 2026/09/23 07:00:55 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo client # copying 1 paths... worker1 # [ 115.401368] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 worker1 # [ 115.419295] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 115.424966] nix-daemon[852]: reaped child process 1142, status = succeeded client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 115.443834] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 115.455858] nix-grpc-daemon[770]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... worker2 # [ 115.471145] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=120 client # [ 115.593429] nix-daemon[1090]: reaped child process 1176, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.76 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 115.766516] nix-daemon[1090]: accepted connection from pid 1195, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 115.931229] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:49186 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 115.938821] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 115.946056] nix-grpc-daemon[902]: time=2026-09-23T07:00:55.568Z level=INFO msg="All 1 paths already cached" worker1 # [ 115.957158] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 115.963266] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 115.990414] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 116.005797] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43424 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker2 # [ 116.027686] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 116.043502] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 116.067364] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5) worker2 # [ 116.091958] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5) worker2 # [ 116.126765] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 315 ms (attempt 3/5) worker2 # [ 116.444913] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 596 ms (attempt 4/5) worker2 # [ 117.043256] nix-grpc-daemon[770]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 117.052784] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 117.060260] nix-grpc-daemon[770]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker2 # [ 117.226983] nix-daemon[842]: accepted connection from pid 1197, user nix-grpc-daemon (trusted) worker2 # [ 117.250480] nix-daemon[842]: reaped child process 1200, status = succeeded lb # [ 117.322801] niks3-server[1015]: 2026/09/23 07:00:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 117.259332] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.890Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 117.261542] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.890Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" worker2 # [ 117.288369] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.920Z level=INFO msg="Uploading 1 narinfos" lb # [ 117.352529] niks3-server[1015]: 2026/09/23 07:00:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign lb # [ 117.368368] niks3-server[1015]: 2026/09/23 07:00:56 INFO Signed narinfos id=31 count=1 lb # [ 117.371613] niks3-server[1015]: 2026/09/23 07:00:57 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 117.390171] niks3-server[1015]: 2026/09/23 07:00:57 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst worker2 # [ 117.323831] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.955Z level=INFO msg="Upload complete. (249ms)" lb # [ 117.398327] niks3-server[1015]: 2026/09/23 07:00:57 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 117.407714] niks3-server[1015]: 2026/09/23 07:00:57 INFO Completed upload id=31 lb # [ 117.411215] niks3-server[1015]: 2026/09/23 07:00:57 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker2 # [ 117.332315] nix-grpc-daemon[770]: ts=2026-09-23T07:00:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 paths=1 nar_bytes_in=568 worker2 # [ 117.343532] nix-grpc-daemon[770]: ts=2026-09-23T07:00:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 117.356308] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 117.361654] nix-daemon[842]: reaped child process 1190, status = succeeded worker2 # [ 117.376991] nix-grpc-daemon[770]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker2 # [ 117.393842] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 118.271309] envoy[895]: conn peer=[2001:db8:1::1]:48200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3887 tx=2182 ms=2475 lb # [ 118.276391] envoy[895]: conn peer=[2001:db8:1::1]:48210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=95 lb # [ 118.281578] envoy[895]: conn peer=[2001:db8:1::1]:48220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4322 tx=2374 ms=1321 lb # [ 118.287219] envoy[895]: conn peer=[2001:db8:1::1]:42596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=407 ms=52 lb # [ 118.292090] envoy[895]: conn peer=[2001:db8:1::1]:42602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3872 tx=2067 ms=2348 lb # [ 118.297757] envoy[895]: conn peer=[2001:db8:1::1]:42616 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=96 worker2 # [ 118.484575] nix-daemon[842]: reaped child process 1205, status = succeeded worker2 # [ 118.585628] nix-daemon[842]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker2 # [ 118.601685] nix-daemon[842]: reaped child process 1224, status = succeeded lb # [ 118.683100] niks3-server[1015]: 2026/09/23 07:00:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 118.619693] nix-grpc-daemon[854]: time=2026-09-23T07:00:58.250Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 118.715713] niks3-server[1015]: 2026/09/23 07:00:58 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls worker2 # [ 118.651185] nix-grpc-daemon[854]: time=2026-09-23T07:00:58.283Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.729011] niks3-server[1015]: 2026/09/23 07:00:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign lb # [ 118.739626] niks3-server[1015]: 2026/09/23 07:00:58 INFO Signed narinfos id=32 count=1 lb # [ 118.743988] niks3-server[1015]: 2026/09/23 07:00:58 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker2 # [ 118.678049] nix-grpc-daemon[854]: time=2026-09-23T07:00:58.310Z level=INFO msg="Upload complete. (269ms)" lb # [ 118.754570] niks3-server[1015]: 2026/09/23 07:00:58 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 118.761539] niks3-server[1015]: 2026/09/23 07:00:58 INFO Completed upload id=32 lb # [ 118.764825] niks3-server[1015]: 2026/09/23 07:00:58 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker2 # [ 118.689103] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 client # copying 1 paths... worker1 # [ 118.714284] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:49186 worker2 # [ 118.707692] nix-daemon[842]: reaped child process 1202, status = succeeded worker1 # [ 118.726786] nix-grpc-daemon[765]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker2 # [ 118.758289] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 worker2 # [ 118.765460] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=120 client # [ 118.842546] nix-daemon[1090]: reaped child process 1198, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.25 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 119.009413] nix-daemon[1090]: accepted connection from pid 1217, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 119.158426] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:43424 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 119.173892] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 119.185465] nix-grpc-daemon[902]: time=2026-09-23T07:00:58.808Z level=INFO msg="All 1 paths already cached" worker1 # [ 119.202331] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker1 # [ 119.211660] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 119.220617] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 119.227920] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 119.238202] nix-grpc-daemon[902]: time=2026-09-23T07:00:58.861Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker1 # [ 119.248357] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 119.265787] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 119.418360] nix-daemon[852]: accepted connection from pid 1172, user nix-grpc-daemon (trusted) worker1 # [ 119.438262] nix-daemon[852]: reaped child process 1175, status = succeeded lb # [ 119.505257] niks3-server[1015]: 2026/09/23 07:00:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.453138] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.076Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 119.455096] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.078Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 119.537981] niks3-server[1015]: 2026/09/23 07:00:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker1 # [ 119.483275] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.106Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.553621] niks3-server[1015]: 2026/09/23 07:00:59 INFO Signed narinfos id=33 count=1 lb # [ 119.559998] niks3-server[1015]: 2026/09/23 07:00:59 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 119.571915] niks3-server[1015]: 2026/09/23 07:00:59 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker1 # [ 119.515942] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.139Z level=INFO msg="Upload complete. (236ms)" lb # [ 119.578361] niks3-server[1015]: 2026/09/23 07:00:59 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete worker1 # [ 119.524138] nix-grpc-daemon[765]: ts=2026-09-23T07:00:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=568 lb # [ 119.589106] niks3-server[1015]: 2026/09/23 07:00:59 INFO Completed upload id=33 lb # [ 119.592250] niks3-server[1015]: 2026/09/23 07:00:59 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker1 # [ 119.540217] nix-grpc-daemon[765]: ts=2026-09-23T07:00:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 119.547864] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 119.553518] nix-daemon[852]: reaped child process 1170, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker1 # [ 119.567455] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 120.860529] nix-daemon[852]: reaped child process 1179, status = succeeded worker1 # [ 120.944349] nix-daemon[852]: accepted connection from pid 1195, user nix-grpc-daemon (trusted) worker1 # [ 120.961160] nix-daemon[852]: reaped child process 1198, status = succeeded lb # [ 121.029129] niks3-server[1015]: 2026/09/23 07:01:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 120.978482] nix-grpc-daemon[902]: time=2026-09-23T07:01:00.600Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 121.062860] niks3-server[1015]: 2026/09/23 07:01:00 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker1 # [ 121.011246] nix-grpc-daemon[902]: time=2026-09-23T07:01:00.634Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.074792] niks3-server[1015]: 2026/09/23 07:01:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 121.086420] niks3-server[1015]: 2026/09/23 07:01:00 INFO Signed narinfos id=34 count=1 lb # [ 121.091628] niks3-server[1015]: 2026/09/23 07:01:00 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 121.102115] niks3-server[1015]: 2026/09/23 07:01:00 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 121.107263] niks3-server[1015]: 2026/09/23 07:01:00 INFO Completed upload id=34 worker1 # [ 121.045643] nix-grpc-daemon[902]: time=2026-09-23T07:01:00.669Z level=INFO msg="Upload complete. (267ms)" lb # [ 121.110367] niks3-server[1015]: 2026/09/23 07:01:00 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo client # copying 1 paths... worker1 # [ 121.057679] nix-grpc-daemon[765]: ts=2026-09-23T07:01:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker2 # [ 121.054699] nix-grpc-daemon[770]: ts=2026-09-23T07:01:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43424 worker1 # [ 121.076508] nix-daemon[852]: reaped child process 1177, status = succeeded client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 121.112290] nix-grpc-daemon[765]: ts=2026-09-23T07:01:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 121.119476] nix-grpc-daemon[765]: ts=2026-09-23T07:01:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=120 client # [ 121.182957] nix-daemon[1090]: reaped child process 1220, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.34 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 8.97 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv worker2 # [ 121.062616] nix-grpc-daemon[770]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.18 seconds) worker2 # [ 122.176827] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 123.056612] nix-grpc-daemon[770]: ts=2026-09-23T07:01:02Z level=info event=signal_exit worker1 # [ 125.061815] nix-grpc-daemon[765]: ts=2026-09-23T07:01:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53432 worker2 # [ 125.051303] nix-grpc-daemon[770]: ts=2026-09-23T07:01:04Z level=info event=scheduler_restarting worker1 # [ 125.069745] nix-grpc-daemon[765]: ts=2026-09-23T07:01:04Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 125.080825] nix-grpc-daemon[770]: ts=2026-09-23T07:01:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 125.108215] nix-daemon[842]: reaped child process 844, status = succeeded worker2 # [ 125.227365] nix-grpc-daemon[770]: E0000 00:00:1790146864.859355 816 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 126.802649] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 126.807506] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 126.813420] systemd[1]: nix-grpc-daemon.service: Consumed 3.609s CPU time over 1min 47.956s wall clock time, 66M memory peak, 93.7K incoming IP traffic, 179.8K outgoing IP traffic. worker2 # [ 126.829245] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 126.835356] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 126.838009] systemd[1]: nix-grpc-daemon.socket: Consumed 623us CPU time over 1min 56.348s wall clock time, 256K memory peak, 104.5K incoming IP traffic, 92.4K outgoing IP traffic. lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.07 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 127.149502] nix-grpc-daemon[765]: ts=2026-09-23T07:01:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49186 worker1 # [ 127.156219] nix-grpc-daemon[765]: ts=2026-09-23T07:01:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:49186 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 127.265963] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 127.351960] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 127.466193] nix-grpc-daemon[765]: ts=2026-09-23T07:01:07Z level=info event=signal_exit worker2 # [ 127.800079] systemd[1]: Starting scheduler-token.service... worker2 # [ 127.891023] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 127.901387] systemd[1]: Finished scheduler-token.service. worker2 # [ 127.903478] systemd[1]: scheduler-token.service: Consumed 57ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 127.911629] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 128.121323] niks3-server[1015]: 2026/09/23 07:01:07 INFO lead: released remote=192.168.1.3:59182 worker2 # [ 128.052471] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 128.054855] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 128.058791] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 128.064057] nix-grpc-daemon[1249]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 128.065519] nix-grpc-daemon[1249]: E0000 00:00:1790146867.695715 1249 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 128.153116] niks3-server[1015]: 2026/09/23 07:01:07 INFO lead: acquired remote=192.168.1.4:33100 worker2 # [ 128.104053] nix-grpc-daemon[1249]: E0000 00:00:1790146867.701978 1249 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 128.106405] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 128.115956] nix-grpc-daemon[1249]: E0000 00:00:1790146867.711310 1292 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 128.119012] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_take_over worker2 # [ 128.120260] nix-grpc-daemon[1249]: E0000 00:00:1790146867.728461 1292 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 128.122528] nix-grpc-daemon[1249]: E0000 00:00:1790146867.728599 1292 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 128.127787] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 128.130815] nix-grpc-daemon[1249]: E0000 00:00:1790146867.747477 1292 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 128.133340] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.134599] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 128.275719] envoy[895]: conn peer=[2001:db8:1::1]:42618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=2151 ms=2861 lb # [ 128.280880] envoy[895]: conn peer=[2001:db8:1::1]:42630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=28 lb # [ 128.285634] envoy[895]: conn peer=[2001:db8:1::1]:42636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2123 ms=1964 lb # [ 128.290475] envoy[895]: conn peer=[2001:db8:1::1]:43648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=28 lb # [ 128.295133] envoy[895]: conn peer=[2001:db8:1::4]:52508 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=19063 tx=1863 ms=91316 lb # [ 128.300153] envoy[895]: conn peer=[2001:db8:1::1]:43652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=523 ms=29 worker2 # [ 128.332394] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.337444] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 128.538404] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.543600] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 128.744342] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.749328] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 128.950211] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.955289] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 129.062341] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=scheduler_restarting lb # [ 129.121029] envoy[895]: [2026-09-23 07:01:08.756][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 129.062727] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 129.071828] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53432 worker1 # [ 129.081629] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 129.112426] nix-daemon[852]: reaped child process 854, status = succeeded worker2 # [ 129.129503] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:51638 worker2 # [ 129.156073] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 129.172138] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 129.176001] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:51644 worker2 # [ 129.192061] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 129.287712] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 129.656475] nix-grpc-daemon[765]: E0000 00:00:1790146869.280063 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true worker1 # [ 130.759002] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 130.763881] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 130.772301] systemd[1]: nix-grpc-daemon.service: Consumed 2.894s CPU time over 1min 52.563s wall clock time, 66.9M memory peak, 82.4K incoming IP traffic, 161.4K outgoing IP traffic. worker1 # [ 130.784634] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 130.786760] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 130.793419] systemd[1]: nix-grpc-daemon.socket: Consumed 695us CPU time over 2min 253ms wall clock time, 256K memory peak, 294.5K incoming IP traffic, 102.5K outgoing IP traffic. lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.08 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 131.082135] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 131.095918] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 131.129547] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5) worker2 # [ 131.143311] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 151 ms (attempt 2/5) worker2 # [ 131.301958] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 3/5) worker2 # [ 131.395975] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 437 ms (attempt 4/5) worker2 # [ 131.835288] nix-grpc-daemon[1249]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 131.841946] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 131.864657] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 131.881809] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 131.910933] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:11Z level=info event=niks3_push_started pid=1310 worker2 # [ 132.057592] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.688Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 132.166681] nix-daemon[842]: accepted connection from pid 1323, user nix-grpc-daemon (trusted) worker2 # [ 132.186268] nix-daemon[842]: reaped child process 1326, status = succeeded lb # [ 132.259209] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 132.268367] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 132.207157] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.837Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 132.210747] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.837Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" worker2 # [ 132.213133] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.837Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" lb # [ 132.329091] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign worker2 # [ 132.269512] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.899Z level=INFO msg="Uploading 3 narinfos" lb # [ 132.344320] niks3-server[1015]: 2026/09/23 07:01:11 INFO Signed narinfos id=35 count=1 lb # [ 132.346443] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 132.362278] niks3-server[1015]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 132.372707] niks3-server[1015]: 2026/09/23 07:01:11 INFO Signed narinfos id=36 count=2 lb # [ 132.374633] niks3-server[1015]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 132.388767] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 132.397222] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 132.403464] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 132.412802] niks3-server[1015]: 2026/09/23 07:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 132.417454] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 132.421078] niks3-server[1015]: 2026/09/23 07:01:12 INFO Completed upload id=35 lb # [ 132.423636] niks3-server[1015]: 2026/09/23 07:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 132.427637] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 132.430509] niks3-server[1015]: 2026/09/23 07:01:12 INFO Completed upload id=36 worker2 # [ 132.357139] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.989Z level=INFO msg="Upload complete. (296ms)" worker2 # [ 132.369109] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 132.386312] nix-daemon[842]: reaped child process 1308, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.54 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 132.519104] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 132.530246] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 132.682167] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 132.684502] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 132.688447] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 132.695332] nix-grpc-daemon[1221]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 132.705538] nix-grpc-daemon[1221]: E0000 00:00:1790146872.316902 1221 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 132.736176] nix-grpc-daemon[1221]: E0000 00:00:1790146872.319018 1221 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.738489] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 132.744796] nix-grpc-daemon[1221]: E0000 00:00:1790146872.329786 1261 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 132.747675] nix-grpc-daemon[1221]: E0000 00:00:1790146872.348980 1261 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.750324] nix-grpc-daemon[1221]: E0000 00:00:1790146872.349155 1261 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 132.752737] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 132.753546] nix-grpc-daemon[1221]: E0000 00:00:1790146872.367722 1261 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.764422] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 132.757130] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:51644 worker2 # [ 132.763965] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.09 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.60 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb # [ 132.980544] systemd[1]: Stopping niks3 server... lb # [ 132.981701] niks3-server[1015]: 2026/09/23 07:01:12 INFO Shutdown signal received, draining in-flight requests timeout=10s worker1 # [ 132.918945] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=scheduler_lock_lost status=200 worker2 # [ 132.916140] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:12Z level=info event=scheduler_lock_lost status=200 lb # [ 133.000337] niks3-server[1015]: 2026/09/23 07:01:12 INFO lead: released remote=192.168.1.4:33100 lb # [ 133.009615] systemd[1]: niks3.service: Deactivated successfully. lb # [ 133.012098] systemd[1]: Stopped niks3 server. lb # [ 133.012838] systemd[1]: niks3.service: Consumed 628ms CPU time over 1min 44.122s wall clock time, 27.6M memory peak, 98.8K incoming IP traffic, 142.3K outgoing IP traffic. lb # [ 133.022589] systemd[1]: Starting niks3 server... lb # [ 133.079595] niks3-server[1210]: 2026/09/23 07:01:12 goose: no migrations to run. current version: 20260920000000 lb # [ 133.086288] niks3-server[1210]: 2026/09/23 07:01:12 OK 1_commit_pending_closure.sql (4.8ms) lb # [ 133.090987] niks3-server[1210]: 2026/09/23 07:01:12 OK 2_object_stats_trigger.sql (4.31ms) lb # [ 133.092978] niks3-server[1210]: 2026/09/23 07:01:12 goose: up to current file version: 2 lb # [ 133.094277] niks3-server[1210]: 2026/09/23 07:01:12 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 133.104854] niks3-server[1210]: 2026/09/23 07:01:12 INFO Read proxy enabled — serving cache objects from S3 lb # [ 133.109851] niks3-server[1210]: 2026/09/23 07:01:12 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 133.111417] niks3-server[1210]: 2026/09/23 07:01:12 INFO systemd watchdog enabled interval=15s lb # [ 133.113563] niks3-server[1210]: 2026/09/23 07:01:12 INFO Starting HTTP server address=0.0.0.0:5751 lb: waiting for unit niks3.service lb # [ 133.115493] systemd[1]: Started niks3 server. lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 133.505664] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40470 worker1 # [ 133.525433] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 133.548233] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 133.566542] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 1/5) worker1 # [ 133.667509] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 124 ms (attempt 2/5) worker1 # [ 133.794510] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 3/5) lb # [ 133.994981] niks3-server[1210]: 2026/09/23 07:01:13 INFO lead: acquired remote=192.168.1.4:51108 worker1 # [ 133.931148] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 263 ms (attempt 4/5) worker1 # [ 134.204352] nix-grpc-daemon[1221]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 134.211991] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 134.248496] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 134.255242] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 134.303593] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 134.321226] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40470 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 134.335523] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 134.350512] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 134.507003] nix-daemon[842]: accepted connection from pid 1345, user nix-grpc-daemon (trusted) worker2 # [ 134.526565] nix-daemon[842]: reaped child process 1348, status = succeeded lb # [ 134.605472] niks3-server[1210]: 2026/09/23 07:01:14 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 134.556229] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.188Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 134.559414] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.190Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 134.658835] niks3-server[1210]: 2026/09/23 07:01:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 134.682022] niks3-server[1210]: 2026/09/23 07:01:14 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 134.700613] niks3-server[1210]: 2026/09/23 07:01:14 INFO Signed narinfos id=37 count=1 worker2 # [ 134.631906] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.262Z level=INFO msg="Uploading 1 narinfos" lb # [ 134.712622] niks3-server[1210]: 2026/09/23 07:01:14 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 134.720399] niks3-server[1210]: 2026/09/23 07:01:14 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 134.727693] niks3-server[1210]: 2026/09/23 07:01:14 INFO Completed upload id=37 worker2 # [ 134.653051] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.285Z level=INFO msg="Upload complete. (285ms)" lb # [ 134.734607] niks3-server[1210]: 2026/09/23 07:01:14 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker2 # [ 134.665079] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 134.686724] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 134.691236] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 134.700305] nix-daemon[842]: reaped child process 1343, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker2 # [ 134.709449] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 135.796573] nix-daemon[842]: reaped child process 1352, status = succeeded worker2 # [ 135.886158] nix-daemon[842]: accepted connection from pid 1368, user nix-grpc-daemon (trusted) worker2 # [ 135.907764] nix-daemon[842]: reaped child process 1371, status = succeeded lb # [ 135.981426] niks3-server[1210]: 2026/09/23 07:01:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 135.923629] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.553Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 135.925693] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.553Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 136.037838] niks3-server[1210]: 2026/09/23 07:01:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign worker2 # [ 135.974062] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.606Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.050022] niks3-server[1210]: 2026/09/23 07:01:15 INFO Signed narinfos id=38 count=1 lb # [ 136.056397] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 136.067554] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 136.077250] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 136.089564] niks3-server[1210]: 2026/09/23 07:01:15 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 136.096188] niks3-server[1210]: 2026/09/23 07:01:15 INFO Completed upload id=38 worker2 # [ 136.022225] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.654Z level=INFO msg="Upload complete. (300ms)" lb # [ 136.099475] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 136.045420] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 136.064377] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 136.069319] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 136.074221] nix-daemon[842]: reaped child process 1350, status = succeeded worker2 # [ 136.087199] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 136.102923] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 136.120258] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.752Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 136.130966] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 136.145388] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 136.300637] nix-daemon[842]: accepted connection from pid 1377, user nix-grpc-daemon (trusted) worker2 # [ 136.318539] nix-daemon[842]: reaped child process 1380, status = succeeded lb # [ 136.396784] niks3-server[1210]: 2026/09/23 07:01:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 136.334670] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.966Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 136.336821] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.966Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 136.431066] niks3-server[1210]: 2026/09/23 07:01:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 136.363776] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.995Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.444629] niks3-server[1210]: 2026/09/23 07:01:16 INFO Signed narinfos id=39 count=1 lb # [ 136.453863] niks3-server[1210]: 2026/09/23 07:01:16 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 136.464563] niks3-server[1210]: 2026/09/23 07:01:16 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst worker2 # [ 136.404857] nix-grpc-daemon[1310]: time=2026-09-23T07:01:16.037Z level=INFO msg="Upload complete. (244ms)" lb # [ 136.480124] niks3-server[1210]: 2026/09/23 07:01:16 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 136.487960] niks3-server[1210]: 2026/09/23 07:01:16 INFO Completed upload id=39 lb # [ 136.490828] niks3-server[1210]: 2026/09/23 07:01:16 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 136.416371] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 136.436255] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 136.440605] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 136.448314] nix-daemon[842]: reaped child process 1375, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker2 # [ 136.458896] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 137.918318] nix-daemon[842]: reaped child process 1384, status = succeeded worker2 # [ 138.028478] nix-daemon[842]: accepted connection from pid 1400, user nix-grpc-daemon (trusted) worker2 # [ 138.048615] nix-daemon[842]: reaped child process 1403, status = succeeded lb # [ 138.121738] niks3-server[1210]: 2026/09/23 07:01:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 138.058718] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 138.060545] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.689Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 138.163761] niks3-server[1210]: 2026/09/23 07:01:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker2 # [ 138.099442] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.731Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.178929] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 138.188588] niks3-server[1210]: 2026/09/23 07:01:17 INFO Signed narinfos id=40 count=1 lb # [ 138.194396] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 138.209274] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker2 # [ 138.142042] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.774Z level=INFO msg="Upload complete. (273ms)" lb # [ 138.216229] niks3-server[1210]: 2026/09/23 07:01:17 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 138.220116] niks3-server[1210]: 2026/09/23 07:01:17 INFO Completed upload id=40 lb # [ 138.224581] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker2 # [ 138.159823] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 138.178654] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 138.183771] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 138.188901] nix-daemon[842]: reaped child process 1382, status = succeeded lb # [ 138.280177] envoy[895]: conn peer=192.168.1.3:49072 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1994 tx=920 ms=94481 lb # [ 138.285091] envoy[895]: conn peer=[2001:db8:1::1]:50456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=282 ms=1303 worker2 # [ 138.222522] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 138.237645] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 138.252432] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.884Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 138.263728] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 138.280361] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 138.427266] nix-daemon[842]: accepted connection from pid 1409, user nix-grpc-daemon (trusted) worker2 # [ 138.444329] nix-daemon[842]: reaped child process 1412, status = succeeded lb # [ 138.522195] niks3-server[1210]: 2026/09/23 07:01:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 138.460164] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.092Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 138.463540] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.095Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 138.556156] niks3-server[1210]: 2026/09/23 07:01:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign worker2 # [ 138.490590] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.122Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.570688] niks3-server[1210]: 2026/09/23 07:01:18 INFO Signed narinfos id=41 count=1 lb # [ 138.580194] niks3-server[1210]: 2026/09/23 07:01:18 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 138.592494] niks3-server[1210]: 2026/09/23 07:01:18 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker2 # [ 138.527278] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.159Z level=INFO msg="Upload complete. (233ms)" lb # [ 138.604086] niks3-server[1210]: 2026/09/23 07:01:18 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 138.608874] niks3-server[1210]: 2026/09/23 07:01:18 INFO Completed upload id=41 worker2 # [ 138.536319] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=1 nar_bytes_in=784 lb # [ 138.612904] niks3-server[1210]: 2026/09/23 07:01:18 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 138.550894] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 worker2 # [ 138.560182] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) worker2 # [ 138.564594] nix-daemon[842]: reaped child process 1407, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 138.579532] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 139.411769] nix-daemon[842]: reaped child process 1416, status = succeeded worker2 # [ 139.519717] nix-daemon[842]: accepted connection from pid 1432, user nix-grpc-daemon (trusted) worker2 # [ 139.535150] nix-daemon[842]: reaped child process 1435, status = succeeded lb # [ 139.617067] niks3-server[1210]: 2026/09/23 07:01:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 139.554930] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 139.556608] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.186Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 139.659168] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 139.596532] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.228Z level=INFO msg="Uploading 1 narinfos" lb # [ 139.673975] niks3-server[1210]: 2026/09/23 07:01:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 139.683805] niks3-server[1210]: 2026/09/23 07:01:19 INFO Signed narinfos id=42 count=1 lb # [ 139.688883] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 139.694927] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 139.703082] niks3-server[1210]: 2026/09/23 07:01:19 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete worker2 # [ 139.629091] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.261Z level=INFO msg="Upload complete. (240ms)" lb # [ 139.706689] niks3-server[1210]: 2026/09/23 07:01:19 INFO Completed upload id=42 lb # [ 139.708668] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 139.644292] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 139.659823] nix-daemon[842]: reaped child process 1414, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.40 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 7.07 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 140.120904] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 141.081452] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:20Z level=info event=signal_exit lb # [ 142.000729] niks3-server[1210]: 2026/09/23 07:01:21 INFO lead: released remote=192.168.1.4:51108 worker2 # [ 142.932709] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=scheduler_restarting worker1 # [ 142.944841] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_restarting addr=lb:50051 lb # [ 143.004158] envoy[895]: [2026-09-23 07:01:22.636][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 142.940552] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 142.946192] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:51644 worker2 # [ 142.951490] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 142.956682] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:51644 worker2 # [ 142.961740] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 142.987425] nix-daemon[842]: reaped child process 1302, status = succeeded worker2 # [ 143.126917] nix-grpc-daemon[1249]: E0000 00:00:1790146882.758917 1295 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker1 # [ 143.144352] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.149816] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 143.154973] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 143.200346] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 143.207054] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 143.215079] systemd[1]: nix-grpc-daemon.service: Consumed 1.372s CPU time over 15.293s wall clock time, 39.8M memory peak, 47.1K incoming IP traffic, 83.5K outgoing IP traffic. worker2 # [ 143.229157] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 143.231047] systemd[1]: Closed Nix worker-protocol over gRPC. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker2 # [ 143.235787] systemd[1]: nix-grpc-daemon.socket: Consumed 857us CPU time over 15.967s wall clock time, 256K memory peak, 271.5K incoming IP traffic, 32.4K outgoing IP traffic. worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 143.350214] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.353937] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1 # [ 143.554613] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.559917] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.760543] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.765626] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 143.926325] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_take_over lb # [ 143.988766] niks3-server[1210]: 2026/09/23 07:01:23 INFO lead: acquired remote=192.168.1.3:35874 worker1 # [ 143.967942] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 143.971939] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 144.168590] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.173609] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 144.375986] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.379969] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 144.577073] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.582412] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 144.782869] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.791707] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:44690 worker1 # [ 144.797427] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 145.730548] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 145.738237] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 145.763558] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 145.770280] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 145.810471] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 145.828179] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53174 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 145.844439] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 145.859971] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 145.891884] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=info event=niks3_push_started pid=1301 worker1 # [ 146.038759] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.662Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 146.148121] nix-daemon[852]: accepted connection from pid 1314, user nix-grpc-daemon (trusted) worker1 # [ 146.169130] nix-daemon[852]: reaped child process 1317, status = succeeded lb # [ 146.229908] niks3-server[1210]: 2026/09/23 07:01:25 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 146.179510] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.802Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 146.184309] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.802Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 146.274595] niks3-server[1210]: 2026/09/23 07:01:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker1 # [ 146.223988] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.846Z level=INFO msg="Uploading 1 narinfos" lb # [ 146.286553] niks3-server[1210]: 2026/09/23 07:01:25 INFO Signed narinfos id=43 count=1 lb # [ 146.306828] niks3-server[1210]: 2026/09/23 07:01:25 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 146.314110] niks3-server[1210]: 2026/09/23 07:01:25 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 146.322344] niks3-server[1210]: 2026/09/23 07:01:25 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 146.329055] niks3-server[1210]: 2026/09/23 07:01:25 INFO Completed upload id=43 worker1 # [ 146.265967] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.889Z level=INFO msg="Upload complete. (223ms)" lb # [ 146.330904] niks3-server[1210]: 2026/09/23 07:01:25 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker1 # [ 146.280325] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 146.292367] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 146.298121] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 146.302738] nix-daemon[852]: reaped child process 1299, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... worker1 # [ 146.315115] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 147.380692] nix-daemon[852]: reaped child process 1321, status = succeeded worker1 # [ 147.457018] nix-daemon[852]: accepted connection from pid 1337, user nix-grpc-daemon (trusted) worker1 # [ 147.474587] nix-daemon[852]: reaped child process 1340, status = succeeded lb # [ 147.543093] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 147.492363] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.114Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 147.493835] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.115Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 147.571335] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 147.596311] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls worker1 # [ 147.542775] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.166Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.606275] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 147.614839] niks3-server[1210]: 2026/09/23 07:01:27 INFO Signed narinfos id=44 count=1 lb # [ 147.619270] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 147.627283] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete worker1 # [ 147.566833] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.190Z level=INFO msg="Upload complete. (242ms)" lb # [ 147.630879] niks3-server[1210]: 2026/09/23 07:01:27 INFO Completed upload id=44 lb # [ 147.633149] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker1 # [ 147.588526] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 147.600259] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 147.603903] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 147.607590] nix-daemon[852]: reaped child process 1319, status = succeeded worker1 # [ 147.631726] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 147.645674] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 147.659489] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.283Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 147.670159] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 147.683545] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 147.837076] nix-daemon[852]: accepted connection from pid 1346, user nix-grpc-daemon (trusted) worker1 # [ 147.854792] nix-daemon[852]: reaped child process 1349, status = succeeded lb # [ 147.919919] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 147.867378] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.490Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 147.869113] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.490Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" worker1 # [ 147.899246] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.522Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.949419] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst lb # [ 147.964647] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign lb # [ 147.975593] niks3-server[1210]: 2026/09/23 07:01:27 INFO Signed narinfos id=45 count=1 lb # [ 147.984349] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 147.987905] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 147.996938] niks3-server[1210]: 2026/09/23 07:01:27 INFO Completed upload id=45 worker1 # [ 147.934404] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.558Z level=INFO msg="Upload complete. (236ms)" lb # [ 147.999666] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker1 # [ 147.944324] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 147.960622] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 147.969977] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 147.973260] nix-daemon[852]: reaped child process 1344, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker1 # [ 147.987798] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) lb # [ 148.275221] envoy[895]: conn peer=[2001:db8:1::1]:50464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9736 tx=4476 ms=6152 lb # [ 148.280428] envoy[895]: conn peer=[2001:db8:1::4]:34528 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5513 tx=665 ms=15018 worker1 # [ 149.239605] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:28Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:44692 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 149.618426] nix-daemon[852]: reaped child process 1353, status = succeeded worker1 # [ 149.712076] nix-daemon[852]: accepted connection from pid 1369, user nix-grpc-daemon (trusted) worker1 # [ 149.731019] nix-daemon[852]: reaped child process 1372, status = succeeded lb # [ 149.792619] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.741737] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.364Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 149.743194] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.364Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 149.827879] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 149.844059] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 149.863584] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 149.871025] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 149.875893] niks3-server[1210]: 2026/09/23 07:01:29 INFO Signed narinfos id=46 count=1 worker1 # [ 149.814331] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.437Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.884859] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 149.896478] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo lb # [ 149.903712] niks3-server[1210]: 2026/09/23 07:01:29 INFO Completed upload id=46 worker1 # [ 149.842481] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.466Z level=INFO msg="Upload complete. (244ms)" worker1 # [ 149.856359] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 149.879428] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 149.884528] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 149.889458] nix-daemon[852]: reaped child process 1351, status = succeeded worker1 # [ 149.902646] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 149.917167] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 149.931827] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.555Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 149.942817] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 149.956134] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 150.105404] nix-daemon[852]: accepted connection from pid 1378, user nix-grpc-daemon (trusted) worker1 # [ 150.121550] nix-daemon[852]: reaped child process 1381, status = succeeded lb # [ 150.190148] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 150.140089] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.763Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 150.141704] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.763Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 150.222124] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign worker1 # [ 150.169735] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.793Z level=INFO msg="Uploading 1 narinfos" lb # [ 150.238537] niks3-server[1210]: 2026/09/23 07:01:29 INFO Signed narinfos id=47 count=1 lb # [ 150.246105] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst lb # [ 150.253188] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 150.266277] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete worker1 # [ 150.201812] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.825Z level=INFO msg="Upload complete. (230ms)" lb # [ 150.272084] niks3-server[1210]: 2026/09/23 07:01:29 INFO Completed upload id=47 lb # [ 150.275288] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 150.212460] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 150.224201] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 150.234131] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 150.237257] nix-daemon[852]: reaped child process 1376, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 150.250935] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 151.561187] nix-daemon[852]: reaped child process 1385, status = succeeded worker1 # [ 151.633094] nix-daemon[852]: accepted connection from pid 1401, user nix-grpc-daemon (trusted) worker1 # [ 151.653861] nix-daemon[852]: reaped child process 1404, status = succeeded lb # [ 151.722122] niks3-server[1210]: 2026/09/23 07:01:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 151.672527] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.294Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 151.674434] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.294Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 151.769662] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker1 # [ 151.721003] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.344Z level=INFO msg="Uploading 1 narinfos" lb # [ 151.786415] niks3-server[1210]: 2026/09/23 07:01:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 151.793705] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 151.799847] niks3-server[1210]: 2026/09/23 07:01:31 INFO Signed narinfos id=48 count=1 lb # [ 151.807379] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls worker1 # [ 151.741427] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.365Z level=INFO msg="Upload complete. (278ms)" lb # [ 151.811277] niks3-server[1210]: 2026/09/23 07:01:31 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 151.815032] niks3-server[1210]: 2026/09/23 07:01:31 INFO Completed upload id=48 lb # [ 151.817426] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 151.753257] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 151.771247] nix-daemon[852]: reaped child process 1383, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.29 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds) worker2 # [ 152.055330] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 152.065488] systemd[1]: Starting scheduler-token.service... worker2 # [ 152.149760] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 152.155137] systemd[1]: Finished scheduler-token.service. worker2 # [ 152.160536] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 152.167148] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 152.290535] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 152.292867] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 152.296712] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 152.301100] nix-grpc-daemon[1456]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 152.323589] nix-grpc-daemon[1456]: E0000 00:00:1790146891.933328 1456 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 152.352491] nix-grpc-daemon[1456]: E0000 00:00:1790146891.937441 1456 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.354795] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 152.358030] nix-grpc-daemon[1456]: E0000 00:00:1790146891.944746 1496 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 152.361089] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 152.362210] nix-grpc-daemon[1456]: E0000 00:00:1790146891.969463 1496 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.364576] nix-grpc-daemon[1456]: E0000 00:00:1790146891.969635 1496 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 152.366958] nix-grpc-daemon[1456]: E0000 00:00:1790146891.989457 1496 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.375749] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 152.397918] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 152.404349] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:44690 worker1 # [ 152.413242] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.81 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 152.743097] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 9c8bd89e06c0430e83a64a1c000ee651 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) client # [ 152.943612] nix[1344]: don't know how to build these paths: client # [ 152.945190] nix[1344]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 152.959928] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 152.972271] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 152.978154] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 153.022766] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 153.039123] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53174 client # [ 153.036269] nix[1344]: copying 1 paths to 'lb:50051'... worker1 # [ 153.054049] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 client # [ 153.038599] nix[1344]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 153.070763] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 153.232983] nix-daemon[852]: accepted connection from pid 1417, user nix-grpc-daemon (trusted) worker1 # [ 153.254123] nix-daemon[852]: reaped child process 1420, status = succeeded lb # [ 153.319799] niks3-server[1210]: 2026/09/23 07:01:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 153.268067] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 153.269755] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.891Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 153.358084] niks3-server[1210]: 2026/09/23 07:01:32 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker1 # [ 153.306422] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.929Z level=INFO msg="Uploading 1 narinfos" lb # [ 153.370052] niks3-server[1210]: 2026/09/23 07:01:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 153.379375] niks3-server[1210]: 2026/09/23 07:01:33 INFO Signed narinfos id=49 count=1 lb # [ 153.387446] niks3-server[1210]: 2026/09/23 07:01:33 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 153.399592] niks3-server[1210]: 2026/09/23 07:01:33 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 153.412134] niks3-server[1210]: 2026/09/23 07:01:33 INFO Completed upload id=49 worker1 # [ 153.349153] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.972Z level=INFO msg="Upload complete. (261ms)" lb # [ 153.414493] niks3-server[1210]: 2026/09/23 07:01:33 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker1 # [ 153.360779] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 153.375508] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 worker1 # [ 153.385904] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 153.390976] nix-daemon[852]: reaped child process 1415, status = succeeded client # [ 153.385204] nix[1344]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker1 # [ 153.404247] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 155.535912] systemd-journald[271]: Received client request to rotate journal, rotating. worker2 # [ 155.556072] systemd-journald[271]: Vacuuming done, freed 0B of archived journals from /var/log/journal/0ca198ba34754993b8a9f9af0e0d185e. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 155.619430] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 worker1 # [ 155.709428] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:35Z level=info event=draining lb # [ 158.277723] envoy[895]: conn peer=[2001:db8:1::1]:38968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4364 ms=6045 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 161.116593] nix-daemon[852]: reaped child process 1424, status = succeeded worker1 # [ 161.193214] nix-daemon[852]: accepted connection from pid 1465, user nix-grpc-daemon (trusted) worker1 # [ 161.213553] nix-daemon[852]: reaped child process 1468, status = succeeded lb # [ 161.279165] niks3-server[1210]: 2026/09/23 07:01:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 161.231460] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.852Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 161.233436] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.852Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 161.327661] niks3-server[1210]: 2026/09/23 07:01:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign worker1 # [ 161.275529] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.899Z level=INFO msg="Uploading 1 narinfos" lb # [ 161.341105] niks3-server[1210]: 2026/09/23 07:01:40 INFO Signed narinfos id=50 count=1 lb # [ 161.354342] niks3-server[1210]: 2026/09/23 07:01:40 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 161.362824] niks3-server[1210]: 2026/09/23 07:01:40 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 161.372788] niks3-server[1210]: 2026/09/23 07:01:41 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 161.378899] niks3-server[1210]: 2026/09/23 07:01:41 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 161.389046] niks3-server[1210]: 2026/09/23 07:01:41 INFO Completed upload id=50 lb # [ 161.392165] niks3-server[1210]: 2026/09/23 07:01:41 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker1 # [ 161.327310] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.951Z level=INFO msg="Upload complete. (302ms)" worker1 # [ 161.341337] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 161.360979] nix-daemon[852]: reaped child process 1422, status = succeeded client # [ 161.354417] systemd[1]: rs.service: Deactivated successfully. client # [ 161.359593] systemd[1]: rs.service: Consumed 191ms CPU time over 8.612s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 161.713465] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:41Z level=info event=signal_exit lb # [ 161.992314] niks3-server[1210]: 2026/09/23 07:01:41 INFO lead: released remote=192.168.1.3:35874 lb # [ 162.388801] niks3-server[1210]: 2026/09/23 07:01:42 INFO lead: acquired remote=192.168.1.4:35736 worker2 # [ 162.320825] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:41Z level=info event=scheduler_take_over worker2 # [ 162.921383] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 162.934802] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=scheduler_restarting lb # [ 162.994447] envoy[895]: [2026-09-23 07:01:42.627][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 162.939186] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:44690 worker1 # [ 162.947323] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 162.952579] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 162.957433] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:44690 worker1 # [ 162.964853] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 162.987176] nix-daemon[852]: reaped child process 1274, status = succeeded worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.17 seconds) client: waiting for success: ! systemctl is-active rs worker2 # [ 163.122767] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 163.128133] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } worker1 # [ 163.161906] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 163.332758] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 163.337491] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker2 # [ 163.531989] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 163.548144] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 163.553772] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:52782 worker2 # [ 163.559779] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 163.830807] nix-grpc-daemon[1221]: E0000 00:00:1790146903.454412 1267 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 23, started true, shutdown true worker1 # [ 163.847934] nix-grpc-daemon[1221]: E0000 00:00:1790146903.471599 1267 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 164.245795] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:52792 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1 # [ 164.551933] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 164.558405] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 164.563815] systemd[1]: nix-grpc-daemon.service: Consumed 1.570s CPU time over 32.026s wall clock time, 39.1M memory peak, 48.3K incoming IP traffic, 99.8K outgoing IP traffic. worker1 # [ 164.581124] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 164.727845] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 164.730244] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 164.733983] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 164.739240] nix-grpc-daemon[1475]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 164.760106] nix-grpc-daemon[1475]: E0000 00:00:1790146904.362237 1475 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 164.762435] nix-grpc-daemon[1475]: E0000 00:00:1790146904.365776 1475 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 164.783427] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 164.785689] nix-grpc-daemon[1475]: E0000 00:00:1790146904.371547 1516 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 164.788210] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 164.789076] nix-grpc-daemon[1475]: E0000 00:00:1790146904.391705 1516 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 164.794328] nix-grpc-daemon[1475]: E0000 00:00:1790146904.391848 1516 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 164.798641] nix-grpc-daemon[1475]: E0000 00:00:1790146904.406248 1516 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 164.801245] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 164.789873] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:52782 worker2 # [ 164.795519] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.95 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 165.691303] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: ff370474f9e443e7b0b853f7a8a58c56 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.08 seconds) worker2 # [ 165.905057] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:52806 worker2 # [ 165.919736] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted) client # [ 165.931717] nix[1380]: don't know how to build these paths: client # [ 165.933443] nix[1380]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 165.950950] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 165.957922] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 166.003422] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52806 worker2 # [ 166.019234] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5) worker2 # [ 166.119601] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 2/5) worker2 # [ 166.176523] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 385 ms (attempt 3/5) worker2 # [ 166.563382] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 770 ms (attempt 4/5) worker2 # [ 167.336924] nix-grpc-daemon[1456]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 167.341847] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 167.363480] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52806 client # [ 167.373206] nix[1380]: copying 1 paths to 'lb:50051'... worker2 # [ 167.379210] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52806 client # [ 167.377667] nix[1380]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker2 # [ 167.394882] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted) worker2 # [ 167.422147] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=info event=niks3_push_started pid=1586 worker2 # [ 167.573613] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.204Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 167.679021] nix-daemon[842]: accepted connection from pid 1599, user nix-grpc-daemon (trusted) worker2 # [ 167.697193] nix-daemon[842]: reaped child process 1602, status = succeeded lb # [ 167.779523] niks3-server[1210]: 2026/09/23 07:01:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 167.717987] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 167.719469] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.349Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 167.820459] niks3-server[1210]: 2026/09/23 07:01:47 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls worker2 # [ 167.757311] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.388Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.834041] niks3-server[1210]: 2026/09/23 07:01:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 167.843629] niks3-server[1210]: 2026/09/23 07:01:47 INFO Signed narinfos id=51 count=1 lb # [ 167.853320] niks3-server[1210]: 2026/09/23 07:01:47 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 167.867182] niks3-server[1210]: 2026/09/23 07:01:47 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 167.875705] niks3-server[1210]: 2026/09/23 07:01:47 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker2 # [ 167.802366] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.434Z level=INFO msg="Upload complete. (229ms)" lb # [ 167.880383] niks3-server[1210]: 2026/09/23 07:01:47 INFO Completed upload id=51 worker2 # [ 167.812277] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52806 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 167.828861] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52806 worker2 # [ 167.837087] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted) worker2 # [ 167.843521] nix-daemon[842]: reaped child process 1584, status = succeeded client # [ 167.850007] nix[1380]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker2 # [ 167.859290] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted) lb # [ 168.284926] envoy[895]: conn peer=[2001:db8:1::1]:55508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1516 ms=8393 lb # [ 168.291046] envoy[895]: conn peer=[2001:db8:1::3]:47676 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2582 tx=921 ms=30392 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 169.556750] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1456 (nix-grpc-daemon) on client request. worker2 # [ 169.558374] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 169.652827] niks3-server[1210]: 2026/09/23 07:01:49 INFO lead: released remote=192.168.1.4:35736 worker2 # [ 169.582545] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 169.590945] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. client # [ 169.580995] nix[1380]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker1 # [ 169.598280] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" lb # [ 169.660474] envoy[895]: [2026-09-23 07:01:49.294][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination} client # [ 169.587381] nix[1380]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 169.597244] systemd[1]: nix-grpc-daemon.service: Consumed 506ms CPU time over 17.412s wall clock time, 34.9M memory peak, 15.6K incoming IP traffic, 28.2K outgoing IP traffic. worker2 # [ 169.599232] systemd[1]: Starting scheduler-token.service... worker2 # [ 169.640313] nix-daemon[842]: reaped child process 1604, status = succeeded worker2 # [ 169.641903] nix-daemon[842]: reaped child process 1572, status = succeeded worker2 # [ 169.648789] nix-daemon[1606]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 169.661690] nix-daemon[842]: reaped child process 1606, status = failed with exit code 1 worker2 # [ 169.675962] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 169.679895] systemd[1]: Finished scheduler-token.service. worker2 # [ 169.682622] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 79ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 169.688175] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 169.689648] systemd[1]: Starting scheduler-token.service... worker2 # [ 169.728908] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 169.732090] systemd[1]: Finished scheduler-token.service. worker2 # [ 169.732883] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 48ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 169.736725] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 169.811639] niks3-server[1210]: 2026/09/23 07:01:49 INFO lead: acquired remote=192.168.1.3:54964 worker1 # [ 169.750494] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_take_over worker1 # [ 169.798379] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 169.837750] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 169.839976] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 169.843694] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 169.849271] nix-grpc-daemon[1649]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 169.850724] nix-grpc-daemon[1649]: E0000 00:00:1790146909.480630 1649 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 169.879948] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds) client: must succeed: sleep 5.0 worker2 # [ 169.892145] nix-grpc-daemon[1649]: E0000 00:00:1790146909.486609 1649 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 169.894442] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 169.903577] nix-grpc-daemon[1649]: E0000 00:00:1790146909.499003 1690 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 169.906603] nix-grpc-daemon[1649]: E0000 00:00:1790146909.516255 1690 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 169.909279] nix-grpc-daemon[1649]: E0000 00:00:1790146909.516452 1690 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 169.911636] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 169.915167] nix-grpc-daemon[1649]: E0000 00:00:1790146909.535074 1690 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 169.920235] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 169.922098] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 170.080830] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.086213] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" client # [ 170.085358] nix[1380]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting worker2 # [ 170.123406] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.130061] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 170.286997] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.292397] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 170.331106] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.337831] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 170.493133] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.498297] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 170.538855] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.544110] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 170.700675] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.704700] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 170.744969] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.750037] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 170.900808] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 170.905799] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 170.952536] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 170.956627] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client # [ 171.088864] nix[1380]: scheduler at lb:50051: no healthy upstream, reconnecting worker1 # [ 171.106526] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 171.111637] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 171.152980] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 171.157990] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 171.312430] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 171.317413] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 171.359022] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 171.364130] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 171.518126] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 171.523264] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 171.565068] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 171.600120] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 171.605218] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45536 worker1 # [ 171.610829] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 171.726472] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 171.730603] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45536 worker1 # [ 171.735689] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 173.116818] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:52Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 173.117189] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:52Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 173.157058] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54922 worker2 # [ 173.172655] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted) worker2 # [ 173.195277] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted) client # [ 173.205369] nix[1380]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker2 # [ 173.214192] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted) worker1 # [ 174.264850] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:53Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:45550 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker2 # [ 175.229979] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:54Z level=info event=niks3_push_started pid=1731 worker2 # [ 175.260266] nix-daemon[842]: reaped child process 1706, status = succeeded worker2 # [ 175.361963] nix-grpc-daemon[1731]: time=2026-09-23T07:01:54.993Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 175.468265] nix-daemon[842]: accepted connection from pid 1744, user nix-grpc-daemon (trusted) worker2 # [ 175.487137] nix-daemon[842]: reaped child process 1747, status = succeeded lb # [ 175.566755] niks3-server[1210]: 2026/09/23 07:01:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 175.504078] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.135Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 175.505556] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.135Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 175.621264] niks3-server[1210]: 2026/09/23 07:01:55 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls worker2 # [ 175.558900] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.190Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.635597] niks3-server[1210]: 2026/09/23 07:01:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 175.653225] niks3-server[1210]: 2026/09/23 07:01:55 INFO Signed narinfos id=52 count=1 lb # [ 175.660479] niks3-server[1210]: 2026/09/23 07:01:55 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 175.671156] niks3-server[1210]: 2026/09/23 07:01:55 INFO Completed upload id=52 worker2 # [ 175.595621] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.227Z level=INFO msg="Upload complete. (230ms)" worker2 # [ 175.607921] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54922 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker2 # [ 175.622630] nix-daemon[842]: reaped child process 1704, status = succeeded client # [ 175.638235] systemd[1]: mid.service: Deactivated successfully. client # [ 175.642134] systemd[1]: mid.service: Consumed 203ms CPU time over 9.946s wall clock time, 43.4M memory peak, 9.3K incoming IP traffic, 11.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.72 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 176.384623] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: bd0bb50dc39d43e9a9cd2817e54d8d76 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 176.563899] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:54922 client # [ 176.564892] nix[1416]: don't know how to build these paths: client # [ 176.566591] nix[1416]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 176.593377] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:56Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 176.599501] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:56Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 176.662109] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 176.677036] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 176.699060] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5) worker1 # [ 176.711348] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 2/5) worker1 # [ 176.737049] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 3/5) worker1 # [ 176.753956] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 779 ms (attempt 4/5) worker1 # [ 177.535708] nix-grpc-daemon[1475]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 177.539620] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 177.559074] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 client # [ 177.556140] nix[1416]: copying 1 paths to 'lb:50051'... worker1 # [ 177.574942] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 client # [ 177.560148] nix[1416]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker1 # [ 177.592498] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 177.633915] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=info event=niks3_push_started pid=1576 worker1 # [ 177.787364] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.409Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 177.897559] nix-daemon[852]: accepted connection from pid 1589, user nix-grpc-daemon (trusted) worker1 # [ 177.916658] nix-daemon[852]: reaped child process 1592, status = succeeded lb # [ 177.986758] niks3-server[1210]: 2026/09/23 07:01:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 177.938891] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.561Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 177.942836] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.561Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 178.028192] niks3-server[1210]: 2026/09/23 07:01:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign worker1 # [ 177.978976] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.601Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.044577] niks3-server[1210]: 2026/09/23 07:01:57 INFO Signed narinfos id=53 count=1 lb # [ 178.052205] niks3-server[1210]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst lb # [ 178.065987] niks3-server[1210]: 2026/09/23 07:01:57 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete worker1 # [ 178.009858] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.633Z level=INFO msg="Upload complete. (219ms)" lb # [ 178.073084] niks3-server[1210]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 178.075524] niks3-server[1210]: 2026/09/23 07:01:57 INFO Completed upload id=53 lb # [ 178.078905] niks3-server[1210]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker1 # [ 178.017271] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 178.028322] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 178.042605] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 178.045142] nix-daemon[852]: reaped child process 1573, status = succeeded client # [ 178.039221] nix[1416]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... worker1 # [ 178.059103] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) lb # [ 178.287209] envoy[895]: conn peer=[2001:db8:1::4]:59704 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3483 tx=421 ms=17222 lb # [ 178.292954] envoy[895]: conn peer=[2001:db8:1::1]:39946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4258 tx=2100 ms=9730 client: must succeed: systemctl kill -s INT intr worker1 # [ 179.010734] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:58Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 178.989910] systemd[1]: intr.service: Sent signal SIGINT to main process 1416 (nix) on client request. client # [ 178.999658] nix[1416]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 179.012486] nix[1416]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds) client # [ 179.037283] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 179.042117] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 179.044352] systemd[1]: intr.service: Consumed 168ms CPU time over 2.655s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 179.097153] nix-daemon[1596]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 179.104623] nix-daemon[852]: reaped child process 1594, status = succeeded worker1 # [ 179.107087] nix-daemon[852]: reaped child process 1596, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 2.80 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 179.211252] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 189f693ebbf147aba338e74d845eb910 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds) worker2 # [ 179.418523] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:54922 client # [ 179.420227] nix[1437]: don't know how to build these paths: client # [ 179.421241] nix[1437]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 179.450172] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:59Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 179.456309] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:59Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 179.483335] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54922 worker2 # [ 179.498861] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5) worker2 # [ 179.515899] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 2/5) worker2 # [ 179.624355] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 356 ms (attempt 3/5) worker2 # [ 179.982177] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 501 ms (attempt 4/5) worker2 # [ 180.490592] nix-grpc-daemon[1649]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 180.497847] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 180.516805] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:54922 client # [ 180.525162] nix[1437]: copying 1 paths to 'lb:50051'... worker2 # [ 180.530219] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54922 client # [ 180.525418] nix[1437]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker2 # [ 180.549991] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted) worker2 # [ 180.689817] nix-daemon[842]: accepted connection from pid 1779, user nix-grpc-daemon (trusted) worker2 # [ 180.711642] nix-daemon[842]: reaped child process 1782, status = succeeded lb # [ 180.787574] niks3-server[1210]: 2026/09/23 07:02:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 180.724355] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.355Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 180.725829] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.356Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 180.819885] niks3-server[1210]: 2026/09/23 07:02:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker2 # [ 180.758886] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.390Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.836773] niks3-server[1210]: 2026/09/23 07:02:00 INFO Signed narinfos id=54 count=1 lb # [ 180.843897] niks3-server[1210]: 2026/09/23 07:02:00 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 180.857274] niks3-server[1210]: 2026/09/23 07:02:00 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst worker2 # [ 180.791329] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.423Z level=INFO msg="Upload complete. (223ms)" lb # [ 180.870573] niks3-server[1210]: 2026/09/23 07:02:00 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 180.875645] niks3-server[1210]: 2026/09/23 07:02:00 INFO Completed upload id=54 worker2 # [ 180.800323] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54922 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 180.879033] niks3-server[1210]: 2026/09/23 07:02:00 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker2 # [ 180.815059] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54922 worker2 # [ 180.821863] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted) worker2 # [ 180.826673] nix-daemon[842]: reaped child process 1777, status = succeeded worker2 # [ 180.837757] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted) client # [ 180.831135] nix[1437]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2 worker2 # Checking switch inhibitors... done worker2 # activating the configuration... worker2 # [ 183.416729] nixos[1818]: switching to system configuration /nix/store/zgy718pd25v31g01575mcapmgw5qlpcy-nixos-system-worker2-test worker2 # [ 183.418514] systemd[1]: Stopped target Local File Systems. worker2 # [ 183.419302] systemd[1]: Stopped target Remote File Systems. worker2 # setting up /etc... worker2 # [ 183.949199] systemd[1]: Reload requested from client PID 1818 ('.switch-to-conf') (unit backdoor.service)... worker2 # [ 183.950690] systemd[1]: Reloading... lb # [ 188.283847] envoy[895]: conn peer=[2001:db8:1::1]:48344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=826 ms=2461 worker2 # [ 188.858363] systemd[1]: Reloading finished in 4901 ms. worker2 # restarting sysinit-reactivation.target worker2 # [ 188.997110] systemd[1]: Starting Re-setup tmpfiles on a system that is already running.... worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service worker2 # [ 189.120259] systemd[1]: Finished Re-setup tmpfiles on a system that is already running.. worker2 # [ 189.121403] systemd[1]: Reached target Reactivate sysinit units. worker2 # [ 189.122193] systemd[1]: Reloading D-Bus System Message Bus... worker2 # [ 189.123225] dbus-broker-launch[492]: Caught SIGHUP, trigger reload. worker2 # [ 189.140280] systemd[1]: Reloading Nix worker-protocol over gRPC... worker2 # [ 189.143604] dbus-broker-launch[492]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 189.148697] dbus-broker-launch[492]: NSS returned no entry for 'systemd-timesync' worker2 # [ 189.150335] dbus-broker-launch[492]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 189.180636] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker2 # [ 189.186463] systemd[1]: Reloaded D-Bus System Message Bus. worker2 # [ 189.196381] systemd[1]: Reached target Remote File Systems. worker2 # [ 189.202273] systemd[1]: Starting linger-users.service... worker2 # [ 189.206499] systemd[1]: Reached target Local File Systems. worker2 # [ 189.207303] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 189.222890] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 189.223728] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 189.225848] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 189.227973] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 189.238397] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 189.239685] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 189.252083] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 189.255293] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 189.274166] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 189.275117] systemd[1]: Finished linger-users.service. worker2 # [ 189.563954] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 189.568212] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service worker2 # [ 189.819082] nixos[1818]: finished switching to system configuration /nix/store/zgy718pd25v31g01575mcapmgw5qlpcy-nixos-system-worker2-test worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.84 seconds) worker2: must succeed: pgrep -f 'read -t [9]0 x' worker2 # [ 189.863708] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:09Z level=info event=draining worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 190.122775] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 190.130637] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 190.155656] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 190.162307] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 190.212886] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 190.227962] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 client # copying 1 paths to 'lb:50051'... worker1 # [ 190.243103] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker1 # [ 190.266599] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 190.425039] nix-daemon[852]: accepted connection from pid 1647, user nix-grpc-daemon (trusted) worker1 # [ 190.445902] nix-daemon[852]: reaped child process 1650, status = succeeded lb # [ 190.513936] niks3-server[1210]: 2026/09/23 07:02:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 190.460400] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.083Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 190.461989] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.083Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 190.544783] niks3-server[1210]: 2026/09/23 07:02:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign worker1 # [ 190.490428] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.113Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.559145] niks3-server[1210]: 2026/09/23 07:02:10 INFO Signed narinfos id=55 count=1 lb # [ 190.566617] niks3-server[1210]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 190.583355] niks3-server[1210]: 2026/09/23 07:02:10 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete worker1 # [ 190.531348] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.155Z level=INFO msg="Upload complete. (247ms)" lb # [ 190.593294] niks3-server[1210]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 190.600645] niks3-server[1210]: 2026/09/23 07:02:10 INFO Completed upload id=55 lb # [ 190.603648] niks3-server[1210]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker1 # [ 190.540332] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=528 worker1 # [ 190.559708] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 190.569856] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 190.573582] nix-daemon[852]: reaped child process 1645, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker1 # [ 190.589353] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker1 # [ 191.627847] nix-daemon[852]: reaped child process 1654, status = succeeded worker1 # [ 191.724426] nix-daemon[852]: accepted connection from pid 1670, user nix-grpc-daemon (trusted) worker1 # [ 191.741655] nix-daemon[852]: reaped child process 1673, status = succeeded lb # [ 191.811409] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.764389] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.765961] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.386Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 191.856680] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign worker1 # [ 191.804673] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.428Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.872400] niks3-server[1210]: 2026/09/23 07:02:11 INFO Signed narinfos id=56 count=1 lb # [ 191.883779] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 191.894317] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 191.901677] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls worker1 # [ 191.850358] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.474Z level=INFO msg="Upload complete. (250ms)" lb # [ 191.912856] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 191.917525] niks3-server[1210]: 2026/09/23 07:02:11 INFO Completed upload id=56 lb # [ 191.923541] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker1 # [ 191.867747] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker1 # [ 191.886166] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 191.890725] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 191.895389] nix-daemon[852]: reaped child process 1652, status = succeeded worker1 # [ 191.922912] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 191.939925] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 191.953740] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.577Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 191.964106] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker1 # [ 191.986679] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 192.144700] nix-daemon[852]: accepted connection from pid 1679, user nix-grpc-daemon (trusted) worker1 # [ 192.164190] nix-daemon[852]: reaped child process 1682, status = succeeded lb # [ 192.224212] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.171468] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.794Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 192.173205] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.794Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 192.256276] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker1 # [ 192.201915] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.825Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.268338] niks3-server[1210]: 2026/09/23 07:02:11 INFO Signed narinfos id=57 count=1 lb # [ 192.271546] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst lb # [ 192.288677] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 192.296550] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls worker1 # [ 192.239785] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.863Z level=INFO msg="Upload complete. (235ms)" lb # [ 192.301618] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo lb # [ 192.305469] niks3-server[1210]: 2026/09/23 07:02:11 INFO Completed upload id=57 worker1 # [ 192.250038] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 192.266030] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 192.274926] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 192.277920] nix-daemon[852]: reaped child process 1677, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker1 # [ 192.292363] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker1 # [ 193.560546] nix-daemon[852]: reaped child process 1686, status = succeeded worker1 # [ 193.661830] nix-daemon[852]: accepted connection from pid 1702, user nix-grpc-daemon (trusted) worker1 # [ 193.678221] nix-daemon[852]: reaped child process 1705, status = succeeded lb # [ 193.748769] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 193.698132] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.320Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 193.699748] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.320Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 193.789564] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign worker1 # [ 193.738148] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.361Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.804484] niks3-server[1210]: 2026/09/23 07:02:13 INFO Signed narinfos id=58 count=1 lb # [ 193.807219] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 193.820977] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 193.824980] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 193.832886] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 193.838122] niks3-server[1210]: 2026/09/23 07:02:13 INFO Completed upload id=58 worker1 # [ 193.775330] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.399Z level=INFO msg="Upload complete. (271ms)" lb # [ 193.842091] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 193.791175] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker1 # [ 193.812454] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker1 # [ 193.817914] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 193.823592] nix-daemon[852]: reaped child process 1684, status = succeeded worker1 # [ 193.850724] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 193.866066] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 193.881091] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.504Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker1 # [ 193.892597] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 193.909271] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 194.061197] nix-daemon[852]: accepted connection from pid 1711, user nix-grpc-daemon (trusted) worker1 # [ 194.079043] nix-daemon[852]: reaped child process 1714, status = succeeded lb # [ 194.146951] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.095635] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.719Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 194.097571] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.719Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 194.174293] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign worker1 # [ 194.120776] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.744Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.189493] niks3-server[1210]: 2026/09/23 07:02:13 INFO Signed narinfos id=59 count=1 lb # [ 194.197535] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 194.212794] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls worker1 # [ 194.151077] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.774Z level=INFO msg="Upload complete. (224ms)" lb # [ 194.218926] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete worker1 # [ 194.160499] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=768 lb # [ 194.224581] niks3-server[1210]: 2026/09/23 07:02:13 INFO Completed upload id=59 lb # [ 194.229613] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker1 # [ 194.175834] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 194.182873] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 194.188869] nix-daemon[852]: reaped child process 1709, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 194.203317] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker1 # [ 195.091917] nix-daemon[852]: reaped child process 1718, status = succeeded worker1 # [ 195.161892] nix-daemon[852]: accepted connection from pid 1734, user nix-grpc-daemon (trusted) worker1 # [ 195.181369] nix-daemon[852]: reaped child process 1737, status = succeeded lb # [ 195.242464] niks3-server[1210]: 2026/09/23 07:02:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 195.192433] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.814Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 195.194891] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.814Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 195.281742] niks3-server[1210]: 2026/09/23 07:02:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker1 # [ 195.229407] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.853Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.295874] niks3-server[1210]: 2026/09/23 07:02:14 INFO Signed narinfos id=60 count=1 lb # [ 195.304295] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 195.313316] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 195.319004] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 195.267425] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.891Z level=INFO msg="Upload complete. (253ms)" lb # [ 195.328749] niks3-server[1210]: 2026/09/23 07:02:14 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 195.333361] niks3-server[1210]: 2026/09/23 07:02:14 INFO Completed upload id=60 lb # [ 195.337081] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker1 # [ 195.281255] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker1 # [ 195.299598] nix-daemon[852]: reaped child process 1716, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.43 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds) worker2: must succeed: pkill -f 'read -t [9]0 x' worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker2 # [ 195.704095] nix-daemon[842]: reaped child process 1786, status = succeeded worker2 # [ 195.837775] nix-daemon[842]: accepted connection from pid 1958, user nix-grpc-daemon (trusted) worker2 # [ 195.856846] nix-daemon[842]: reaped child process 1961, status = succeeded lb # [ 195.934899] niks3-server[1210]: 2026/09/23 07:02:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 195.871587] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.502Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 195.873518] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.502Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 195.970970] niks3-server[1210]: 2026/09/23 07:02:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign worker2 # [ 195.905772] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.537Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.986926] niks3-server[1210]: 2026/09/23 07:02:15 INFO Signed narinfos id=61 count=1 lb # [ 195.992583] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 196.001095] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 196.010259] niks3-server[1210]: 2026/09/23 07:02:15 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete worker2 # [ 195.935968] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.568Z level=INFO msg="Upload complete. (268ms)" lb # [ 196.016777] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 196.021858] niks3-server[1210]: 2026/09/23 07:02:15 INFO Completed upload id=61 lb # [ 196.024390] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker2 # [ 195.949214] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54922 duration_s=15 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 worker2 # [ 195.966009] nix-daemon[842]: reaped child process 1784, status = succeeded client # [ 195.985069] systemd[1]: sw.service: Deactivated successfully. client # [ 195.989492] systemd[1]: sw.service: Consumed 200ms CPU time over 16.777s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.06 seconds) lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 196.866107] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:16Z level=info event=signal_exit lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 197.891818] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45536 worker2 # [ 197.880994] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:17Z level=info event=scheduler_restarting worker1 # [ 197.899999] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:17Z level=info event=worker_gone addr=192.168.1.4:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 197.907680] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 197.927623] nix-daemon[842]: reaped child process 1702, status = succeeded lb # [ 198.284749] envoy[895]: conn peer=[2001:db8:1::1]:51426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9578 tx=4383 ms=5178 lb # [ 198.289789] envoy[895]: conn peer=[2001:db8:1::1]:48350 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3021 tx=1561 ms=16551 lb # [ 198.294668] envoy[895]: conn peer=[2001:db8:1::4]:52180 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7599 tx=679 ms=27971 worker2 # [ 198.433300] nix-grpc-daemon[1649]: E0000 00:00:1790146938.065321 1695 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 199.897242] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 199.903491] systemd[1]: nix-grpc-daemon.service: Consumed 784ms CPU time over 30.318s wall clock time, 36.1M memory peak, 22.6K incoming IP traffic, 36.7K outgoing IP traffic. lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2 # [ 200.892166] systemd[1]: Starting scheduler-token.service... worker2 # [ 200.976402] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 200.984773] systemd[1]: Finished scheduler-token.service. worker2 # [ 200.986777] systemd[1]: scheduler-token.service: Consumed 56ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 201.007530] systemd[1]: Starting Nix worker-protocol over gRPC... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2 # [ 201.118987] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 201.121293] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 201.125058] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 201.130291] nix-grpc-daemon[1976]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 201.131720] nix-grpc-daemon[1976]: E0000 00:00:1790146940.761719 1976 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 201.175510] nix-grpc-daemon[1976]: E0000 00:00:1790146940.767663 1976 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 201.178434] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 201.180279] nix-grpc-daemon[1976]: E0000 00:00:1790146940.775749 2016 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 201.182590] nix-grpc-daemon[1976]: E0000 00:00:1790146940.792600 2016 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 201.187654] nix-grpc-daemon[1976]: E0000 00:00:1790146940.792732 2016 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 201.192067] nix-grpc-daemon[1976]: E0000 00:00:1790146940.807003 2016 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 201.194576] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 201.195412] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 201.207869] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45536 worker1 # [ 201.217091] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 5.40 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 23.12 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 202.499382] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: 35ae9d3fa2b0405395cb14da9678d671 client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1 # [ 202.723052] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45562 client # [ 202.712168] nix[1493]: don't know how to build these paths: client # [ 202.713905] nix[1493]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv client # [ 202.715151] nix[1493]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv client # [ 202.716898] nix[1493]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv client # [ 202.718149] nix[1493]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker1 # [ 202.743565] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=0 worker1 # [ 202.750683] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 202.756962] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=0 worker1 # [ 202.762689] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 202.768073] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=0 worker1 # [ 202.772293] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=0 worker1 # [ 202.791105] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker2 # [ 202.785870] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker1 # [ 202.807134] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker2 # [ 202.799441] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) client # [ 202.804249] nix[1493]: copying 1 paths to 'lb:50051'... worker1 # [ 202.821662] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 client # [ 202.806563] nix[1493]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'... worker1 # [ 202.838201] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker2 # [ 202.824080] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5) worker2 # [ 202.920794] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5) worker2 # [ 202.958622] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 267 ms (attempt 3/5) worker1 # [ 202.987520] nix-daemon[852]: accepted connection from pid 1752, user nix-grpc-daemon (trusted) worker1 # [ 203.005935] nix-daemon[852]: reaped child process 1755, status = succeeded lb # [ 203.075223] niks3-server[1210]: 2026/09/23 07:02:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 203.023705] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.646Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 203.025451] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.646Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)" lb # [ 203.108609] niks3-server[1210]: 2026/09/23 07:02:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 203.056452] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.680Z level=INFO msg="Uploading 1 narinfos" lb # [ 203.120625] niks3-server[1210]: 2026/09/23 07:02:22 INFO Signed narinfos id=62 count=1 lb # [ 203.129960] niks3-server[1210]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls lb # [ 203.146399] niks3-server[1210]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst lb # [ 203.160183] niks3-server[1210]: 2026/09/23 07:02:22 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 203.167365] niks3-server[1210]: 2026/09/23 07:02:22 INFO Completed upload id=62 worker1 # [ 203.104638] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.728Z level=INFO msg="Upload complete. (251ms)" lb # [ 203.174831] niks3-server[1210]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo worker1 # [ 203.116358] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 203.132817] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 203.140543] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 203.147076] nix-daemon[852]: reaped child process 1750, status = succeeded client # [ 203.140961] nix[1493]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051... worker1 # [ 203.160166] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker2 # [ 203.227620] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 4/5) worker2 # [ 203.371208] nix-grpc-daemon[1976]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 203.376923] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 203.394437] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 client # [ 203.404251] nix[1493]: copying 1 paths to 'lb:50051'... worker2 # [ 203.409819] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 client # [ 203.406556] nix[1493]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'... worker2 # [ 203.426187] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 203.451497] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=info event=niks3_push_started pid=2037 worker2 # [ 203.598681] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.230Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 203.703170] nix-daemon[842]: accepted connection from pid 2050, user nix-grpc-daemon (trusted) worker2 # [ 203.721562] nix-daemon[842]: reaped child process 2053, status = succeeded lb # [ 203.799378] niks3-server[1210]: 2026/09/23 07:02:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 203.738576] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.369Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 203.742058] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.369Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)" lb # [ 203.854204] niks3-server[1210]: 2026/09/23 07:02:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign worker2 # [ 203.786714] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.417Z level=INFO msg="Uploading 1 narinfos" lb # [ 203.865043] niks3-server[1210]: 2026/09/23 07:02:23 INFO Signed narinfos id=63 count=1 lb # [ 203.870914] niks3-server[1210]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls lb # [ 203.877516] niks3-server[1210]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst lb # [ 203.888449] niks3-server[1210]: 2026/09/23 07:02:23 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete worker2 # [ 203.819745] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.451Z level=INFO msg="Upload complete. (217ms)" lb # [ 203.899420] niks3-server[1210]: 2026/09/23 07:02:23 INFO Completed upload id=63 worker2 # [ 203.829821] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 203.903703] niks3-server[1210]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.13 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 203.840475] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 203.851466] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 203.856262] nix-daemon[842]: reaped child process 2035, status = succeeded worker2 # [ 203.865362] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) client # [ 203.858155] nix[1493]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051... lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) worker1 # [ 205.415065] nix-daemon[852]: reaped child process 1759, status = succeeded worker1 # [ 205.542499] nix-daemon[852]: accepted connection from pid 1787, user nix-grpc-daemon (trusted) worker1 # [ 205.562318] nix-daemon[852]: reaped child process 1790, status = succeeded lb # [ 205.629258] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 205.560954] nix-daemon[842]: reaped child process 2057, status = succeeded worker1 # [ 205.580098] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 205.581899] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.200Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)" worker1 # [ 205.619585] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.243Z level=INFO msg="Uploading 1 narinfos" lb # [ 205.671996] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv lb # [ 205.685408] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 205.696105] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=64 count=1 lb # [ 205.703694] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst worker1 # [ 205.647760] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.271Z level=INFO msg="Upload complete. (272ms)" lb # [ 205.711487] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls lb # [ 205.714743] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 205.719348] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=64 worker2 # [ 205.649634] nix-daemon[842]: accepted connection from pid 2082, user nix-grpc-daemon (trusted) lb # [ 205.722024] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo worker1 # [ 205.661485] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=1 worker1 # [ 205.676324] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 205.668333] nix-daemon[842]: reaped child process 2085, status = succeeded worker1 # [ 205.681481] nix-daemon[852]: reaped child process 1757, status = succeeded lb # [ 205.745736] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 205.682329] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.312Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 205.683961] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.312Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)" worker1 # [ 205.703010] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 lb # [ 205.781107] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv lb # [ 205.791438] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst worker1 # [ 205.728246] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker2 # [ 205.726940] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.358Z level=INFO msg="Uploading 1 narinfos" lb # [ 205.802039] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 205.810702] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=65 count=1 client # [ 205.733117] nix[1493]: copying 1 paths to 'lb:50051'... worker1 # [ 205.749283] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 client # [ 205.733362] nix[1493]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'... lb # [ 205.816270] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls lb # [ 205.824119] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete worker1 # [ 205.763028] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) lb # [ 205.830052] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=65 lb # [ 205.832639] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo worker2 # [ 205.756272] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.388Z level=INFO msg="Upload complete. (278ms)" worker1 # [ 205.775826] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 205.773152] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=1 worker2 # [ 205.788936] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 205.799865] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 205.805324] nix-daemon[842]: reaped child process 2055, status = succeeded client # [ 205.800790] nix[1493]: copying 1 paths to 'lb:50051'... worker2 # [ 205.811326] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 client # [ 205.806268] nix[1493]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'... worker2 # [ 205.825091] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker1 # [ 205.924762] nix-daemon[852]: accepted connection from pid 1794, user nix-grpc-daemon (trusted) worker1 # [ 205.945643] nix-daemon[852]: reaped child process 1797, status = succeeded lb # [ 206.005718] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 205.952811] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.576Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 205.954289] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.576Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)" worker2 # [ 205.959385] nix-daemon[842]: accepted connection from pid 2089, user nix-grpc-daemon (trusted) lb # [ 206.033463] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign worker1 # [ 205.977727] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.601Z level=INFO msg="Uploading 1 narinfos" worker2 # [ 205.974733] nix-daemon[842]: reaped child process 2092, status = succeeded lb # [ 206.047220] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=66 count=1 lb # [ 206.058722] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst worker2 # [ 205.991500] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.623Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 205.993159] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.623Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)" lb # [ 206.068317] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 206.074662] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls lb # [ 206.083861] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete worker2 # [ 206.018935] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.650Z level=INFO msg="Uploading 1 narinfos" worker1 # [ 206.029157] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.652Z level=INFO msg="Upload complete. (242ms)" lb # [ 206.093421] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=66 lb # [ 206.095501] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign worker1 # [ 206.040352] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 206.109712] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=67 count=1 lb # [ 206.111960] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls worker1 # [ 206.053260] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 lb # [ 206.118852] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo lb # [ 206.126192] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst worker2 # [ 206.056585] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.688Z level=INFO msg="Upload complete. (222ms)" lb # [ 206.131004] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete worker1 # [ 206.070342] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 206.074822] nix-daemon[852]: reaped child process 1792, status = succeeded lb # [ 206.136242] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo lb # [ 206.140199] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=67 worker2 # [ 206.067228] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=416 client # [ 206.069989] nix[1493]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051... worker1 # [ 206.089414] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker2 # [ 206.080570] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 206.089429] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 206.092488] nix-daemon[842]: reaped child process 2087, status = succeeded client # [ 206.096692] nix[1493]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051... worker2 # [ 206.106015] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker1 # [ 207.730243] nix-daemon[852]: reaped child process 1801, status = succeeded worker2 # [ 207.731772] nix-daemon[842]: reaped child process 2096, status = succeeded worker2 # [ 207.827428] nix-daemon[842]: accepted connection from pid 2121, user nix-grpc-daemon (trusted) worker1 # [ 207.841220] nix-daemon[852]: accepted connection from pid 1826, user nix-grpc-daemon (trusted) worker2 # [ 207.843616] nix-daemon[842]: reaped child process 2124, status = succeeded worker1 # [ 207.860685] nix-daemon[852]: reaped child process 1829, status = succeeded lb # [ 207.925025] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 207.862751] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.494Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 207.864400] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.494Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)" worker1 # [ 207.880000] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.500Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 207.881571] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.500Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)" lb # [ 207.931319] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 207.988259] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker2 # [ 207.928810] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.560Z level=INFO msg="Uploading 1 narinfos" worker1 # [ 207.945161] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.568Z level=INFO msg="Uploading 1 narinfos" lb # [ 208.011758] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 208.021186] niks3-server[1210]: 2026/09/23 07:02:27 INFO Signed narinfos id=68 count=1 lb # [ 208.023344] niks3-server[1210]: 2026/09/23 07:02:27 INFO Signed narinfos id=69 count=1 lb # [ 208.029849] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls worker2 # [ 207.965999] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.598Z level=INFO msg="Upload complete. (249ms)" lb # [ 208.038715] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst lb # [ 208.049221] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker2 # [ 207.977103] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=1 lb # [ 208.057732] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst worker1 # [ 207.996533] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.620Z level=INFO msg="Upload complete. (292ms)" lb # [ 208.062849] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 208.066603] niks3-server[1210]: 2026/09/23 07:02:27 INFO Completed upload id=68 lb # [ 208.068719] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls lb # [ 208.071725] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo worker2 # [ 207.998238] nix-daemon[842]: reaped child process 2094, status = succeeded lb # [ 208.075598] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv lb # [ 208.078569] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 208.081801] niks3-server[1210]: 2026/09/23 07:02:27 INFO Completed upload id=69 lb # [ 208.083496] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo worker1 # [ 208.012993] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=1 worker1 # [ 208.029423] nix-daemon[852]: reaped child process 1799, status = succeeded client # [ 208.028520] systemd[1]: burst.service: Deactivated successfully. client # [ 208.034077] systemd[1]: burst.service: Consumed 221ms CPU time over 5.530s wall clock time, 53.6M memory peak, 14.2K incoming IP traffic, 18.4K outgoing IP traffic. lb # [ 208.286582] envoy[895]: conn peer=[2001:db8:1::1]:34700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9654 tx=5168 ms=5302 client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 6.91 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker1 # [ 209.410717] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45562 worker2 # [ 209.403246] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 worker1 # [ 209.453083] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker1 # [ 209.459814] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 209.466538] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker1 # [ 209.471831] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 209.499100] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 209.514172] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 209.528880] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 209.543442] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 209.695777] nix-daemon[852]: accepted connection from pid 1842, user nix-grpc-daemon (trusted) worker1 # [ 209.716873] nix-daemon[852]: reaped child process 1845, status = succeeded lb # [ 209.777539] niks3-server[1210]: 2026/09/23 07:02:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 209.726798] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 209.728459] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.349Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 209.813035] niks3-server[1210]: 2026/09/23 07:02:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign worker1 # [ 209.758360] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.381Z level=INFO msg="Uploading 1 narinfos" lb # [ 209.824005] niks3-server[1210]: 2026/09/23 07:02:29 INFO Signed narinfos id=70 count=1 lb # [ 209.826721] niks3-server[1210]: 2026/09/23 07:02:29 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 209.780842] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.404Z level=INFO msg="Upload complete. (221ms)" lb # [ 209.839189] niks3-server[1210]: 2026/09/23 07:02:29 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls lb # [ 209.848250] niks3-server[1210]: 2026/09/23 07:02:29 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 209.852582] niks3-server[1210]: 2026/09/23 07:02:29 INFO Completed upload id=70 worker1 # [ 209.788271] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 209.854645] niks3-server[1210]: 2026/09/23 07:02:29 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 209.802808] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 209.811818] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 209.814623] nix-daemon[852]: reaped child process 1840, status = succeeded worker1 # [ 209.828129] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 211.326367] nix-daemon[852]: reaped child process 1849, status = succeeded worker1 # [ 211.397793] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 211.413621] nix-daemon[852]: reaped child process 1868, status = succeeded lb # [ 211.481956] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 211.430700] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.053Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 211.432910] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.053Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 211.524219] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst worker1 # [ 211.476711] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.100Z level=INFO msg="Uploading 1 narinfos" lb # [ 211.539208] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 211.548324] niks3-server[1210]: 2026/09/23 07:02:31 INFO Signed narinfos id=71 count=1 lb # [ 211.550526] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 211.560629] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 211.563928] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 211.568683] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo lb # [ 211.572607] niks3-server[1210]: 2026/09/23 07:02:31 INFO Completed upload id=71 worker1 # [ 211.510719] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.134Z level=INFO msg="Upload complete. (271ms)" worker1 # [ 211.529205] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=1 worker1 # [ 211.548884] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 211.558448] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 211.564401] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 211.576880] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 211.587233] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 211.593703] nix-daemon[852]: reaped child process 1847, status = succeeded worker1 # [ 211.598552] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 211.716567] nix-daemon[852]: accepted connection from pid 1874, user nix-grpc-daemon (trusted) worker1 # [ 211.732830] nix-daemon[852]: reaped child process 1877, status = succeeded lb # [ 211.799566] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 211.746319] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.369Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 211.747784] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.369Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 211.828087] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 211.772148] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.395Z level=INFO msg="Uploading 1 narinfos" lb # [ 211.842573] niks3-server[1210]: 2026/09/23 07:02:31 INFO Signed narinfos id=72 count=1 lb # [ 211.850175] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls worker1 # [ 211.801081] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.424Z level=INFO msg="Upload complete. (208ms)" lb # [ 211.863158] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst lb # [ 211.868269] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete worker1 # [ 211.808144] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 211.875431] niks3-server[1210]: 2026/09/23 07:02:31 INFO Completed upload id=72 lb # [ 211.877870] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 211.821730] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 211.827342] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 211.831839] nix-daemon[852]: reaped child process 1872, status = succeeded worker1 # [ 211.841758] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 213.121197] nix-daemon[852]: reaped child process 1881, status = succeeded worker1 # [ 213.230646] nix-daemon[852]: accepted connection from pid 1897, user nix-grpc-daemon (trusted) worker1 # [ 213.247935] nix-daemon[852]: reaped child process 1900, status = succeeded lb # [ 213.316946] niks3-server[1210]: 2026/09/23 07:02:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 213.268428] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 213.270714] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.888Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 213.359258] niks3-server[1210]: 2026/09/23 07:02:32 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls worker1 # [ 213.308922] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.932Z level=INFO msg="Uploading 1 narinfos" lb # [ 213.371161] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 213.382709] niks3-server[1210]: 2026/09/23 07:02:33 INFO Signed narinfos id=73 count=1 lb # [ 213.387445] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 213.395838] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker1 # [ 213.342974] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.966Z level=INFO msg="Upload complete. (285ms)" lb # [ 213.405917] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 213.409676] niks3-server[1210]: 2026/09/23 07:02:33 INFO Completed upload id=73 lb # [ 213.414562] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 213.362745] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=1 worker1 # [ 213.378929] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 213.384595] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 213.389793] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 213.394720] nix-daemon[852]: reaped child process 1879, status = succeeded worker1 # [ 213.396632] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 213.404545] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 213.414633] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 213.546585] nix-daemon[852]: accepted connection from pid 1906, user nix-grpc-daemon (trusted) worker1 # [ 213.567041] nix-daemon[852]: reaped child process 1909, status = succeeded lb # [ 213.629497] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 213.577046] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 213.578519] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.200Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 213.660622] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls worker1 # [ 213.611018] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.234Z level=INFO msg="Uploading 1 narinfos" lb # [ 213.673661] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 213.685190] niks3-server[1210]: 2026/09/23 07:02:33 INFO Signed narinfos id=74 count=1 lb # [ 213.687818] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 213.700831] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 213.705044] niks3-server[1210]: 2026/09/23 07:02:33 INFO Completed upload id=74 worker1 # [ 213.643210] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.266Z level=INFO msg="Upload complete. (219ms)" lb # [ 213.708262] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 213.652310] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 213.667121] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 213.675741] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 213.678928] nix-daemon[852]: reaped child process 1904, status = succeeded worker1 # [ 213.690117] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 214.351282] nix-daemon[852]: reaped child process 1913, status = succeeded worker1 # [ 214.482575] nix-daemon[852]: accepted connection from pid 1929, user nix-grpc-daemon (trusted) worker1 # [ 214.500192] nix-daemon[852]: reaped child process 1932, status = succeeded lb # [ 214.569903] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 214.516432] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.139Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 214.518064] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.139Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 214.614437] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst worker1 # [ 214.564753] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.188Z level=INFO msg="Uploading 1 narinfos" lb # [ 214.628428] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 214.640465] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 214.649979] niks3-server[1210]: 2026/09/23 07:02:34 INFO Signed narinfos id=75 count=1 worker1 # [ 214.598239] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.221Z level=INFO msg="Upload complete. (296ms)" lb # [ 214.660555] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 214.665375] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 214.673536] niks3-server[1210]: 2026/09/23 07:02:34 INFO Completed upload id=75 lb # [ 214.676517] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker1 # [ 214.614301] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=1 worker1 # [ 214.637579] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 214.644728] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 214.650858] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 214.657342] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 214.662907] nix-daemon[852]: reaped child process 1911, status = succeeded worker1 # [ 214.665419] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.285Z level=INFO msg="All 1 paths already cached" worker1 # [ 214.680244] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 214.683747] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 214.815532] nix-daemon[852]: accepted connection from pid 1938, user nix-grpc-daemon (trusted) worker1 # [ 214.830077] nix-daemon[852]: reaped child process 1941, status = succeeded lb # [ 214.899028] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 214.847527] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.471Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 214.849223] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.471Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 214.932225] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker1 # [ 214.880717] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.504Z level=INFO msg="Uploading 1 narinfos" lb # [ 214.945971] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 214.960349] niks3-server[1210]: 2026/09/23 07:02:34 INFO Signed narinfos id=76 count=1 worker1 # [ 214.898928] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.522Z level=INFO msg="Upload complete. (208ms)" lb # [ 214.964475] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 214.906513] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 214.971942] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 214.976970] niks3-server[1210]: 2026/09/23 07:02:34 INFO Completed upload id=76 lb # [ 214.979422] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 214.919646] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 214.925944] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 214.930917] nix-daemon[852]: reaped child process 1936, status = succeeded worker1 # [ 214.941861] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 216.408923] nix-daemon[852]: reaped child process 1945, status = succeeded worker1 # [ 216.522773] nix-daemon[852]: accepted connection from pid 1961, user nix-grpc-daemon (trusted) worker1 # [ 216.542536] nix-daemon[852]: reaped child process 1964, status = succeeded lb # [ 216.608834] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 216.560428] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.181Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 216.562154] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.181Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 216.666138] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst worker1 # [ 216.613004] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.236Z level=INFO msg="Uploading 1 narinfos" lb # [ 216.678763] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 216.687068] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign lb # [ 216.697735] niks3-server[1210]: 2026/09/23 07:02:36 INFO Signed narinfos id=77 count=1 worker1 # [ 216.640526] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.264Z level=INFO msg="Upload complete. (286ms)" lb # [ 216.703380] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 216.707533] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 216.712446] niks3-server[1210]: 2026/09/23 07:02:36 INFO Completed upload id=77 lb # [ 216.715640] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 216.657714] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=1 worker1 # [ 216.678897] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 216.685552] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 216.691264] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 216.696880] nix-daemon[852]: reaped child process 1943, status = succeeded worker1 # [ 216.700854] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 216.706062] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.328Z level=INFO msg="All 1 paths already cached" worker1 # [ 216.710165] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 216.726614] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 216.858905] nix-daemon[852]: accepted connection from pid 1970, user nix-grpc-daemon (trusted) worker1 # [ 216.874988] nix-daemon[852]: reaped child process 1973, status = succeeded lb # [ 216.941738] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 216.890944] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.514Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 216.892516] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.514Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 216.974022] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 216.923600] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.547Z level=INFO msg="Uploading 1 narinfos" lb # [ 216.990572] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign lb # [ 216.999535] niks3-server[1210]: 2026/09/23 07:02:36 INFO Signed narinfos id=78 count=1 worker1 # [ 216.945202] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.569Z level=INFO msg="Upload complete. (213ms)" lb # [ 217.008518] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls worker1 # [ 216.951643] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 217.016819] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 217.020986] niks3-server[1210]: 2026/09/23 07:02:36 INFO Completed upload id=78 lb # [ 217.023669] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 216.964414] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 216.971000] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 216.975704] nix-daemon[852]: reaped child process 1968, status = succeeded worker1 # [ 216.986023] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 217.843162] nix-daemon[852]: reaped child process 1977, status = succeeded worker1 # [ 217.975931] nix-daemon[852]: accepted connection from pid 1993, user nix-grpc-daemon (trusted) worker1 # [ 217.991934] nix-daemon[852]: reaped child process 1996, status = succeeded lb # [ 218.061906] niks3-server[1210]: 2026/09/23 07:02:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 218.008391] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.631Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 218.010014] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.631Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 218.101744] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 218.117427] niks3-server[1210]: 2026/09/23 07:02:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign worker1 # [ 218.063077] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.686Z level=INFO msg="Uploading 1 narinfos" lb # [ 218.128885] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 218.140140] niks3-server[1210]: 2026/09/23 07:02:37 INFO Signed narinfos id=79 count=1 lb # [ 218.145270] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls worker1 # [ 218.093368] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.717Z level=INFO msg="Upload complete. (296ms)" lb # [ 218.155623] niks3-server[1210]: 2026/09/23 07:02:37 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 218.162085] niks3-server[1210]: 2026/09/23 07:02:37 INFO Completed upload id=79 lb # [ 218.169462] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 218.109831] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=1 worker1 # [ 218.130043] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 218.140793] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 218.146011] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 218.151285] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 218.156326] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.779Z level=INFO msg="All 1 paths already cached" worker1 # [ 218.159101] nix-daemon[852]: reaped child process 1975, status = succeeded worker1 # [ 218.168275] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 218.170621] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 218.304651] nix-daemon[852]: accepted connection from pid 2002, user nix-grpc-daemon (trusted) worker1 # [ 218.321907] nix-daemon[852]: reaped child process 2005, status = succeeded lb # [ 218.389768] niks3-server[1210]: 2026/09/23 07:02:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 218.337276] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.960Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 218.339061] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.960Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 218.419016] niks3-server[1210]: 2026/09/23 07:02:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign worker1 # [ 218.364107] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.987Z level=INFO msg="Uploading 1 narinfos" lb # [ 218.433980] niks3-server[1210]: 2026/09/23 07:02:38 INFO Signed narinfos id=80 count=1 lb # [ 218.441005] niks3-server[1210]: 2026/09/23 07:02:38 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 218.393214] nix-grpc-daemon[1576]: time=2026-09-23T07:02:38.016Z level=INFO msg="Upload complete. (211ms)" lb # [ 218.455557] niks3-server[1210]: 2026/09/23 07:02:38 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls lb # [ 218.461864] niks3-server[1210]: 2026/09/23 07:02:38 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 218.466373] niks3-server[1210]: 2026/09/23 07:02:38 INFO Completed upload id=80 worker1 # [ 218.404263] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 218.469473] niks3-server[1210]: 2026/09/23 07:02:38 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 218.414529] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 218.423715] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 218.426561] nix-daemon[852]: reaped child process 2000, status = succeeded worker1 # [ 218.437851] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 219.708229] nix-daemon[852]: reaped child process 2009, status = succeeded worker1 # [ 219.802417] nix-daemon[852]: accepted connection from pid 2025, user nix-grpc-daemon (trusted) worker1 # [ 219.821124] nix-daemon[852]: reaped child process 2028, status = succeeded lb # [ 219.881791] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 219.832224] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.453Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 219.833810] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.453Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 219.922275] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign worker1 # [ 219.870560] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.494Z level=INFO msg="Uploading 1 narinfos" lb # [ 219.936379] niks3-server[1210]: 2026/09/23 07:02:39 INFO Signed narinfos id=81 count=1 lb # [ 219.944272] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 219.955818] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst worker1 # [ 219.900747] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.524Z level=INFO msg="Upload complete. (250ms)" lb # [ 219.966747] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 219.972659] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 219.977513] niks3-server[1210]: 2026/09/23 07:02:39 INFO Completed upload id=81 lb # [ 219.981491] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 219.918979] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=1 worker1 # [ 219.935457] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 219.941546] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 219.947550] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 219.953724] nix-daemon[852]: reaped child process 2007, status = succeeded worker1 # [ 219.955894] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 219.962896] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.585Z level=INFO msg="All 2 paths already cached" worker1 # [ 219.966745] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 219.980142] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 220.109941] nix-daemon[852]: accepted connection from pid 2034, user nix-grpc-daemon (trusted) worker1 # [ 220.126793] nix-daemon[852]: reaped child process 2037, status = succeeded lb # [ 220.195175] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 220.142368] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.765Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 220.143818] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.765Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 220.223777] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 220.171860] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.795Z level=INFO msg="Uploading 1 narinfos" lb # [ 220.236338] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 220.248988] niks3-server[1210]: 2026/09/23 07:02:39 INFO Signed narinfos id=82 count=1 lb # [ 220.251631] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst lb # [ 220.261956] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete worker1 # [ 220.204474] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.828Z level=INFO msg="Upload complete. (214ms)" lb # [ 220.268752] niks3-server[1210]: 2026/09/23 07:02:39 INFO Completed upload id=82 lb # [ 220.272232] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 220.212299] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 220.225837] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 220.234150] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 220.237186] nix-daemon[852]: reaped child process 2032, status = succeeded worker1 # [ 220.249543] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 220.924400] nix-daemon[852]: reaped child process 2041, status = succeeded worker1 # [ 220.990105] nix-daemon[852]: accepted connection from pid 2057, user nix-grpc-daemon (trusted) worker1 # [ 221.007206] nix-daemon[852]: reaped child process 2060, status = succeeded lb # [ 221.075613] niks3-server[1210]: 2026/09/23 07:02:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 221.024350] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.647Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 221.025822] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.647Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" worker1 # [ 221.067749] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.691Z level=INFO msg="Uploading 1 narinfos" lb # [ 221.120743] niks3-server[1210]: 2026/09/23 07:02:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 221.133277] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 221.143046] niks3-server[1210]: 2026/09/23 07:02:40 INFO Signed narinfos id=83 count=1 lb # [ 221.147867] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 221.153354] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 221.162058] niks3-server[1210]: 2026/09/23 07:02:40 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 221.166519] niks3-server[1210]: 2026/09/23 07:02:40 INFO Completed upload id=83 lb # [ 221.169625] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker1 # [ 221.104680] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.728Z level=INFO msg="Upload complete. (246ms)" worker1 # [ 221.121287] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=1 worker1 # [ 221.136091] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 221.140724] nix-daemon[852]: reaped child process 2039, status = succeeded worker1 # [ 221.159068] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 221.175353] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 221.190962] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.814Z level=INFO msg="All 2 paths already cached" worker1 # [ 221.201738] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 221.215445] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 221.369906] nix-daemon[852]: accepted connection from pid 2064, user nix-grpc-daemon (trusted) worker1 # [ 221.389731] nix-daemon[852]: reaped child process 2067, status = succeeded lb # [ 221.456450] niks3-server[1210]: 2026/09/23 07:02:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 221.404096] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.027Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 221.405744] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.027Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 221.489279] niks3-server[1210]: 2026/09/23 07:02:41 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 221.439136] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.062Z level=INFO msg="Uploading 1 narinfos" lb # [ 221.504085] niks3-server[1210]: 2026/09/23 07:02:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign lb # [ 221.513536] niks3-server[1210]: 2026/09/23 07:02:41 INFO Signed narinfos id=84 count=1 lb # [ 221.519264] niks3-server[1210]: 2026/09/23 07:02:41 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 221.529761] niks3-server[1210]: 2026/09/23 07:02:41 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete worker1 # [ 221.474112] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.097Z level=INFO msg="Upload complete. (242ms)" lb # [ 221.538347] niks3-server[1210]: 2026/09/23 07:02:41 INFO Completed upload id=84 lb # [ 221.542932] niks3-server[1210]: 2026/09/23 07:02:41 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 221.484342] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 221.504158] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 221.511015] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 221.516944] nix-daemon[852]: reaped child process 2062, status = succeeded worker1 # [ 221.531233] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 222.992740] nix-daemon[852]: reaped child process 2071, status = succeeded worker1 # [ 223.072877] nix-daemon[852]: accepted connection from pid 2087, user nix-grpc-daemon (trusted) worker1 # [ 223.088788] nix-daemon[852]: reaped child process 2090, status = succeeded lb # [ 223.156752] niks3-server[1210]: 2026/09/23 07:02:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 223.106225] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.728Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 223.107718] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.728Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" worker1 # [ 223.145661] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.769Z level=INFO msg="Uploading 1 narinfos" lb # [ 223.197807] niks3-server[1210]: 2026/09/23 07:02:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign lb # [ 223.209966] niks3-server[1210]: 2026/09/23 07:02:42 INFO Signed narinfos id=85 count=1 lb # [ 223.217945] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 223.228220] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 223.242776] niks3-server[1210]: 2026/09/23 07:02:42 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 223.248668] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst worker1 # [ 223.192455] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.816Z level=INFO msg="Upload complete. (249ms)" lb # [ 223.255820] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo lb # [ 223.259710] niks3-server[1210]: 2026/09/23 07:02:42 INFO Completed upload id=85 worker1 # [ 223.208569] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=1 worker1 # [ 223.223183] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 223.228720] nix-daemon[852]: reaped child process 2069, status = succeeded worker1 # [ 223.247280] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 223.261935] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 223.275622] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.899Z level=INFO msg="All 2 paths already cached" worker1 # [ 223.285690] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 223.298996] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 223.445320] nix-daemon[852]: accepted connection from pid 2094, user nix-grpc-daemon (trusted) worker1 # [ 223.463937] nix-daemon[852]: reaped child process 2097, status = succeeded lb # [ 223.523720] niks3-server[1210]: 2026/09/23 07:02:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 223.471653] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.095Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 223.473364] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.095Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 223.554197] niks3-server[1210]: 2026/09/23 07:02:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign worker1 # [ 223.500579] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.124Z level=INFO msg="Uploading 1 narinfos" lb # [ 223.568736] niks3-server[1210]: 2026/09/23 07:02:43 INFO Signed narinfos id=86 count=1 lb # [ 223.576285] niks3-server[1210]: 2026/09/23 07:02:43 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 223.587298] niks3-server[1210]: 2026/09/23 07:02:43 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 223.533322] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.157Z level=INFO msg="Upload complete. (219ms)" lb # [ 223.595621] niks3-server[1210]: 2026/09/23 07:02:43 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete lb # [ 223.604507] niks3-server[1210]: 2026/09/23 07:02:43 INFO Completed upload id=86 worker1 # [ 223.543210] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 223.609815] niks3-server[1210]: 2026/09/23 07:02:43 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 223.558203] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 worker1 # [ 223.566985] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 223.570466] nix-daemon[852]: reaped child process 2092, status = succeeded worker1 # [ 223.580163] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker1 # [ 224.685802] nix-daemon[852]: reaped child process 2101, status = succeeded worker1 # [ 224.757765] nix-daemon[852]: accepted connection from pid 2117, user nix-grpc-daemon (trusted) worker1 # [ 224.778646] nix-daemon[852]: reaped child process 2120, status = succeeded lb # [ 224.838832] niks3-server[1210]: 2026/09/23 07:02:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 224.787786] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.410Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 224.789606] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.410Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 224.877728] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker1 # [ 224.830403] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.453Z level=INFO msg="Uploading 1 narinfos" lb # [ 224.891899] niks3-server[1210]: 2026/09/23 07:02:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign lb # [ 224.902639] niks3-server[1210]: 2026/09/23 07:02:44 INFO Signed narinfos id=87 count=1 lb # [ 224.908639] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 224.919517] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 224.864187] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.487Z level=INFO msg="Upload complete. (272ms)" lb # [ 224.926552] niks3-server[1210]: 2026/09/23 07:02:44 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete lb # [ 224.931593] niks3-server[1210]: 2026/09/23 07:02:44 INFO Completed upload id=87 lb # [ 224.935174] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 224.876994] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=1 worker1 # [ 224.896892] nix-daemon[852]: reaped child process 2099, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.77 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 225.071963] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 9b7017b0bf964a1aa976a7c939334019 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 225.277720] nix[1563]: don't know how to build these paths: client # [ 225.278625] nix[1563]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2 # [ 225.282500] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49830 client # [ 225.304323] nix[1563]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker worker1 # [ 225.320696] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:44Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 worker2 # [ 225.290930] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 228.293176] envoy[895]: conn peer=[2001:db8:1::1]:34716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27097 tx=12330 ms=15497 client # [ 255.305824] nix[1563]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 255.317855] nix[1563]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 255.354896] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 255.363048] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 255.366906] systemd[1]: gpu.service: Consumed 195ms CPU time over 30.288s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.05 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.54 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1 # [ 256.793743] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:16Z level=info event=unhealthy reason=min_free worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.25 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 257.231523] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 257.240568] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 worker1 # [ 257.277660] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:16Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker1 # [ 257.284306] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:16Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 257.310998] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 257.328500] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 257.344841] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 257.360969] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 257.522814] nix-daemon[842]: accepted connection from pid 2132, user nix-grpc-daemon (trusted) worker2 # [ 257.543329] nix-daemon[842]: reaped child process 2135, status = succeeded lb # [ 257.626346] niks3-server[1210]: 2026/09/23 07:03:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 257.562178] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.194Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 257.564605] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.196Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" worker2 # [ 257.602301] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.234Z level=INFO msg="Uploading 1 narinfos" lb # [ 257.664231] niks3-server[1210]: 2026/09/23 07:03:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign lb # [ 257.680938] niks3-server[1210]: 2026/09/23 07:03:17 INFO Signed narinfos id=88 count=1 lb # [ 257.688284] niks3-server[1210]: 2026/09/23 07:03:17 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst lb # [ 257.702100] niks3-server[1210]: 2026/09/23 07:03:17 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 257.713610] niks3-server[1210]: 2026/09/23 07:03:17 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete lb # [ 257.721778] niks3-server[1210]: 2026/09/23 07:03:17 INFO Completed upload id=88 worker2 # [ 257.646421] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.278Z level=INFO msg="Upload complete. (268ms)" worker2 # [ 257.656587] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 257.741305] niks3-server[1210]: 2026/09/23 07:03:17 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 257.671520] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 257.684844] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 257.690449] nix-daemon[842]: reaped child process 2130, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... worker2 # [ 257.702069] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 258.292078] envoy[895]: conn peer=[2001:db8:1::1]:56396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=30059 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 259.213623] nix-daemon[842]: reaped child process 2139, status = succeeded worker2 # [ 259.283854] nix-daemon[842]: accepted connection from pid 2155, user nix-grpc-daemon (trusted) worker2 # [ 259.306752] nix-daemon[842]: reaped child process 2158, status = succeeded lb # [ 259.379300] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 259.319759] nix-grpc-daemon[2037]: time=2026-09-23T07:03:18.950Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 259.321519] nix-grpc-daemon[2037]: time=2026-09-23T07:03:18.950Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 259.419322] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign worker2 # [ 259.354384] nix-grpc-daemon[2037]: time=2026-09-23T07:03:18.986Z level=INFO msg="Uploading 1 narinfos" lb # [ 259.433497] niks3-server[1210]: 2026/09/23 07:03:19 INFO Signed narinfos id=89 count=1 lb # [ 259.435922] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 259.447028] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 259.459174] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls worker2 # [ 259.389575] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.021Z level=INFO msg="Upload complete. (274ms)" lb # [ 259.467081] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete lb # [ 259.472057] niks3-server[1210]: 2026/09/23 07:03:19 INFO Completed upload id=89 lb # [ 259.474643] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 259.401125] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=1 worker1 # [ 259.422109] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:19Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker1 # [ 259.427815] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:19Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 259.419787] nix-daemon[842]: reaped child process 2137, status = succeeded worker2 # [ 259.454855] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 259.471520] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 259.486668] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.118Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 259.497927] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 259.511749] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 259.661281] nix-daemon[842]: accepted connection from pid 2162, user nix-grpc-daemon (trusted) worker2 # [ 259.680993] nix-daemon[842]: reaped child process 2165, status = succeeded lb # [ 259.758438] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 259.695438] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.327Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 259.697209] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.327Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 259.787453] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 259.724820] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.356Z level=INFO msg="Uploading 1 narinfos" lb # [ 259.798171] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign lb # [ 259.805940] niks3-server[1210]: 2026/09/23 07:03:19 INFO Signed narinfos id=90 count=1 lb # [ 259.810892] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 259.816779] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete lb # [ 259.824545] niks3-server[1210]: 2026/09/23 07:03:19 INFO Completed upload id=90 worker2 # [ 259.751090] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.383Z level=INFO msg="Upload complete. (223ms)" lb # [ 259.827078] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 259.760151] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 259.773421] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 259.779335] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 259.783820] nix-daemon[842]: reaped child process 2160, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 259.794254] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 261.457628] nix-daemon[842]: reaped child process 2169, status = succeeded worker2 # [ 261.563706] nix-daemon[842]: accepted connection from pid 2185, user nix-grpc-daemon (trusted) worker2 # [ 261.584579] nix-daemon[842]: reaped child process 2188, status = succeeded lb # [ 261.662024] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 261.596361] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.228Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 261.597824] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.228Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" worker2 # [ 261.639730] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.271Z level=INFO msg="Uploading 1 narinfos" lb # [ 261.705177] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign lb # [ 261.721300] niks3-server[1210]: 2026/09/23 07:03:21 INFO Signed narinfos id=91 count=1 lb # [ 261.731432] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 261.741137] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 261.749854] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls worker2 # [ 261.683535] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.315Z level=INFO msg="Upload complete. (276ms)" lb # [ 261.759281] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete lb # [ 261.765491] niks3-server[1210]: 2026/09/23 07:03:21 INFO Completed upload id=91 lb # [ 261.770447] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 261.697135] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=1 worker1 # [ 261.718578] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:21Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker1 # [ 261.718944] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:21Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 261.716229] nix-daemon[842]: reaped child process 2167, status = succeeded worker2 # [ 261.751797] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 261.766133] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 261.780498] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.412Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 261.791522] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 261.805013] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 261.959070] nix-daemon[842]: accepted connection from pid 2192, user nix-grpc-daemon (trusted) worker2 # [ 261.977866] nix-daemon[842]: reaped child process 2195, status = succeeded lb # [ 262.049332] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 261.984167] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.616Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 261.985652] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.616Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 262.082443] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 262.017562] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.649Z level=INFO msg="Uploading 1 narinfos" lb # [ 262.096062] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign lb # [ 262.106545] niks3-server[1210]: 2026/09/23 07:03:21 INFO Signed narinfos id=92 count=1 worker2 # [ 262.040811] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.672Z level=INFO msg="Upload complete. (219ms)" lb # [ 262.114666] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst lb # [ 262.123761] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete worker2 # [ 262.050154] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 262.128322] niks3-server[1210]: 2026/09/23 07:03:21 INFO Completed upload id=92 lb # [ 262.130257] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 262.063334] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 worker2 # [ 262.071852] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) worker2 # [ 262.079817] nix-daemon[842]: reaped child process 2190, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 262.085636] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted) client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 263.580434] nix-daemon[842]: reaped child process 2199, status = succeeded worker2 # [ 263.666974] nix-daemon[842]: accepted connection from pid 2215, user nix-grpc-daemon (trusted) worker2 # [ 263.682364] nix-daemon[842]: reaped child process 2218, status = succeeded lb # [ 263.765955] niks3-server[1210]: 2026/09/23 07:03:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 263.703558] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.332Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 263.705565] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.332Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 263.803513] niks3-server[1210]: 2026/09/23 07:03:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign worker2 # [ 263.741653] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.373Z level=INFO msg="Uploading 1 narinfos" lb # [ 263.821177] niks3-server[1210]: 2026/09/23 07:03:23 INFO Signed narinfos id=93 count=1 lb # [ 263.823854] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 263.839342] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 263.770816] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.402Z level=INFO msg="Upload complete. (272ms)" lb # [ 263.847563] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 263.850930] niks3-server[1210]: 2026/09/23 07:03:23 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete lb # [ 263.855287] niks3-server[1210]: 2026/09/23 07:03:23 INFO Completed upload id=93 lb # [ 263.857847] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 263.785113] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=1 worker2 # [ 263.801123] nix-daemon[842]: reaped child process 2197, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.82 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 264.798402] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:24Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.63 seconds) (finished: run the VM test script, in 266.25 seconds) test script finished in 266.51s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.06 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh