tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm client # Disk image does not exist, creating the virtualisation disk image... client # Formatting '/build/vm-state-client/tmp.ZuhhBwFpVt', fmt=raw size=1073741824 client # mke2fs 1.47.4 (6-Mar-2025) lb # Disk image does not exist, creating the virtualisation disk image... worker2 # Disk image does not exist, creating the virtualisation disk image... lb: QEMU running (pid 49) client: QEMU running (pid 50) worker1: QEMU running (pid 52) client # Discarding device blocks: 0/262144 done worker1 # Disk image does not exist, creating the virtualisation disk image... worker2: QEMU running (pid 51) client # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Formatting '/build/vm-state-worker1/tmp.0U3O84ZunV', fmt=raw size=1073741824 lb # Formatting '/build/vm-state-lb/tmp.uM7Oduvt1P', fmt=raw size=1073741824 worker2 # Formatting '/build/vm-state-worker2/tmp.oSagt32KKb', fmt=raw size=1073741824 client # Filesystem UUID: d4dd8969-73c0-43cf-ab04-047d2883e63d (finished: start all VMs, in 0.53 seconds) worker2 # mke2fs 1.47.4 (6-Mar-2025) lb # mke2fs 1.47.4 (6-Mar-2025) worker1 # mke2fs 1.47.4 (6-Mar-2025) client # Superblock backups stored on blocks: lb # Discarding device blocks: 0/262144 done worker1 # Discarding device blocks: 0/262144 done client: waiting for unit network-addresses-eth1.service worker2 # Discarding device blocks: 0/262144 done client # 32768, 98304, 163840, 229376 lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client # lb # Filesystem UUID: 263eff33-f374-436f-b58d-fd57b4da33c1 worker2 # Filesystem UUID: 47eff782-01e5-4cab-8c65-8fec002074a1 client # Allocating group tables: 0/8 done lb # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: client # Writing inode tables: 0/8 done lb # 32768, 98304, 163840, 229376 worker2 # 32768, 98304, 163840, 229376 client # Creating journal (8192 blocks): done lb # worker2 # client # Writing superblocks and filesystem accounting information: 0/8 done lb # Allocating group tables: 0/8 done worker2 # Allocating group tables: 0/8 done client # lb # Writing inode tables: 0/8 done worker2 # Writing inode tables: 0/8 done client # Virtualisation disk image created. lb # Creating journal (8192 blocks): done worker2 # Creating journal (8192 blocks): done client: waiting for the VM to finish booting worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes lb # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Filesystem UUID: bf9ef386-e4ac-4152-8954-4575120e86c0 lb # worker1 # Superblock backups stored on blocks: lb # Virtualisation disk image created. worker1 # 32768, 98304, 163840, 229376 worker2 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # worker2 # worker1 # Allocating group tables: 0/8 done worker2 # Virtualisation disk image created. worker1 # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # worker1 # Virtualisation disk image created. client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 client # [ 0.000000] KASLR enabled client # [ 0.000000] random: crng init done client # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] efi: UEFI not found. client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty client # [ 0.000000] Normal empty client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node client # [ 0.000000] Early memory node ranges client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 client # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test/init regInfo=/nix/store/a9vnspps9sb5b5czzzz1l864fbjlmq8d-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/a9vnspps9sb5b5czzzz1l864fbjlmq8d-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 worker2 # [ 0.000000] KASLR enabled lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] random: crng init done lb # [ 0.000000] KASLR enabled lb # [ 0.000000] random: crng init done worker2 # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] efi: UEFI not found. client # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Zone ranges: lb # [ 0.000000] Zone ranges: worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Policy zone: DMA lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty lb # [ 0.000000] DMA32 empty worker2 # [ 0.000000] Normal empty client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] Normal empty worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026 worker2 # [ 0.000000] Device empty worker1 # [ 0.000000] KASLR enabled lb # [ 0.000000] Device empty worker2 # [ 0.000000] Movable zone start for each node client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] random: crng init done lb # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] Machine model: linux,dummy-virt lb # [ 0.000000] Early memory node ranges client # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Zone ranges: client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] DMA32 empty worker2 # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] Normal empty worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] Device empty lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] Movable zone start for each node client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296 worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fhjd99g778w0n7n58v9pngxdn87svwlf-nixos-system-worker2-test/init regInfo=/nix/store/8clhlxs2384kagmjxsysmqfk4aqyyqmr-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a9n40kyqx09p2mx9kz6ck1ddzmmfmfqv-nixos-system-lb-test/init regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/8clhlxs2384kagmjxsysmqfk4aqyyqmr-closure-info/registration", will be passed to user space. worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration", will be passed to user space. client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h490qvdfgh8rdlrafpm8wgj56x94xnlb-nixos-system-worker1-test/init regInfo=/nix/store/pznpw35xiyr2l3gwkj88qbrfsnnwgdih-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/pznpw35xiyr2l3gwkj88qbrfsnnwgdih-closure-info/registration", will be passed to user space. lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000033] arm-pv: using stolen time PV worker2 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Policy zone: DMA client # [ 0.000653] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] Policy zone: DMA client # [ 0.000834] Console: colour dummy device 80x25 worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000841] printk: legacy console [tty0] enabled lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.001058] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages client # [ 0.001065] pid_max: default: 32768 minimum: 301 lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.001141] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.001315] landlock: Up and running. lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.001317] Yama: becoming mindful. worker1 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.001955] LSM support for eBPF active worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.002118] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.002153] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.003282] cacheinfo: Unable to detect cache hierarchy for CPU 0 worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.004093] rcu: Hierarchical SRCU implementation. worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.004098] rcu: Max phase no-delay instances is 1000. lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.005332] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.005423] EFI services will not be available. lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005533] smp: Bringing up secondary CPUs ... worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005542] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005545] SMP: Total of 1 processors activated. worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005551] CPU: All CPU(s) started at EL1 lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.005563] CPU features: detected: Branch Target Identification worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.005568] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.005571] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.005575] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005578] CPU features: detected: Common not Private translations worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.005581] CPU features: detected: CRC32 instructions worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.005584] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.005588] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.005591] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.005594] CPU features: detected: E0PD worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.005597] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.005600] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.005603] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.005607] CPU features: detected: Fine Grained Traps worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.005611] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.005616] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.005619] CPU features: detected: LSE atomic instructions worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000032] arm-pv: using stolen time PV lb # [ 0.000031] arm-pv: using stolen time PV client # [ 0.005622] CPU features: detected: Privileged Access Never worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.005625] CPU features: detected: PMUv3 worker2 # [ 0.000416] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000434] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.005627] CPU features: detected: RAS Extension Support worker2 # [ 0.000582] Console: colour dummy device 80x25 lb # [ 0.000607] Console: colour dummy device 80x25 worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.005630] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.000589] printk: legacy console [tty0] enabled lb # [ 0.000615] printk: legacy console [tty0] enabled client # [ 0.005633] CPU features: detected: Random Number Generator worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.005635] CPU features: detected: Speculation barrier (SB) lb # [ 0.000810] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000782] pid_max: default: 32768 minimum: 301 client # [ 0.005639] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000816] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000850] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.005641] CPU features: detected: TLB range maintenance instructions lb # [ 0.000893] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000990] landlock: Up and running. worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.001014] landlock: Up and running. worker2 # [ 0.000992] Yama: becoming mindful. client # [ 0.005646] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.001017] Yama: becoming mindful. worker2 # [ 0.001445] LSM support for eBPF active worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.001489] LSM support for eBPF active client # [ 0.005687] alternatives: applying system-wide alternatives worker1 # [ 0.000030] arm-pv: using stolen time PV worker2 # [ 0.001578] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.001631] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.008750] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.001610] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000412] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.001664] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000593] Console: colour dummy device 80x25 worker2 # [ 0.002794] cacheinfo: Unable to detect cache hierarchy for CPU 0 lb # [ 0.002830] cacheinfo: Unable to detect cache hierarchy for CPU 0 client # [ 0.008968] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved) worker1 # [ 0.000602] printk: legacy console [tty0] enabled worker2 # [ 0.003489] rcu: Hierarchical SRCU implementation. client # [ 0.009371] devtmpfs: initialized lb # [ 0.003546] rcu: Hierarchical SRCU implementation. worker2 # [ 0.003493] rcu: Max phase no-delay instances is 1000. lb # [ 0.003550] rcu: Max phase no-delay instances is 1000. client # [ 0.011226] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.000794] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.004662] fsl-mc MSI: its@8080000 domain created lb # [ 0.004720] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.000801] pid_max: default: 32768 minimum: 301 worker2 # [ 0.004748] EFI services will not be available. lb # [ 0.004819] EFI services will not be available. client # [ 0.011250] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.004832] smp: Bringing up secondary CPUs ... worker1 # [ 0.000876] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.004888] smp: Bringing up secondary CPUs ... worker2 # [ 0.004839] smp: Brought up 1 node, 1 CPU worker1 # [ 0.001050] landlock: Up and running. client # [ 0.011464] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.004896] smp: Brought up 1 node, 1 CPU worker1 # [ 0.001053] Yama: becoming mindful. worker2 # [ 0.004842] SMP: Total of 1 processors activated. client # [ 0.011469] 0 pages in range for non-PLT usage lb # [ 0.004899] SMP: Total of 1 processors activated. worker1 # [ 0.001502] LSM support for eBPF active worker2 # [ 0.004849] CPU: All CPU(s) started at EL1 client # [ 0.011470] 508288 pages in range for PLT usage lb # [ 0.004904] CPU: All CPU(s) started at EL1 worker2 # [ 0.004860] CPU features: detected: Branch Target Identification client # [ 0.011567] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.001639] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.004915] CPU features: detected: Branch Target Identification client # [ 0.012311] DMI not present or invalid. worker2 # [ 0.004865] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.004920] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.001671] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.016182] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.002710] cacheinfo: Unable to detect cache hierarchy for CPU 0 worker2 # [ 0.004868] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.004923] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.018481] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.003431] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004872] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.003434] rcu: Max phase no-delay instances is 1000. client # [ 0.018660] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.004926] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.004641] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.004876] CPU features: detected: Common not Private translations lb # [ 0.004930] CPU features: detected: Common not Private translations client # [ 0.018865] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.004731] EFI services will not be available. worker2 # [ 0.004879] CPU features: detected: CRC32 instructions lb # [ 0.004933] CPU features: detected: CRC32 instructions worker1 # [ 0.004810] smp: Bringing up secondary CPUs ... client # [ 0.018887] audit: initializing netlink subsys (disabled) worker1 # [ 0.004818] smp: Brought up 1 node, 1 CPU worker2 # [ 0.004881] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.004936] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.019498] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.004821] SMP: Total of 1 processors activated. worker2 # [ 0.004885] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.004940] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.019500] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.004827] CPU: All CPU(s) started at EL1 worker2 # [ 0.004888] CPU features: detected: Data independent timing control (DIT) client # [ 0.019504] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.004838] CPU features: detected: Branch Target Identification lb # [ 0.004943] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.004891] CPU features: detected: E0PD lb # [ 0.004946] CPU features: detected: E0PD client # [ 0.019510] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.004843] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.004949] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004893] CPU features: detected: Enhanced Counter Virtualization client # [ 0.019512] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.004847] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.004896] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.004952] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.019539] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.004850] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.004900] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.004955] CPU features: detected: Enhanced Virtualization Traps client # [ 0.019548] cpuidle: using governor ladder client # [ 0.019554] cpuidle: using governor menu lb # [ 0.004958] CPU features: detected: Fine Grained Traps worker2 # [ 0.004903] CPU features: detected: Fine Grained Traps worker1 # [ 0.004854] CPU features: detected: Common not Private translations worker1 # [ 0.004857] CPU features: detected: CRC32 instructions client # [ 0.019764] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.004906] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.004963] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.019781] ASID allocator initialised with 65536 entries worker1 # [ 0.004860] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.004911] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.004968] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.021076] Serial: AMBA PL011 UART driver worker2 # [ 0.004914] CPU features: detected: LSE atomic instructions lb # [ 0.004971] CPU features: detected: LSE atomic instructions worker1 # [ 0.004864] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.026389] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.004917] CPU features: detected: Privileged Access Never lb # [ 0.004974] CPU features: detected: Privileged Access Never worker1 # [ 0.004867] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.004920] CPU features: detected: PMUv3 client # [ 0.026542] printk: console [ttyAMA0] enabled lb # [ 0.004976] CPU features: detected: PMUv3 worker1 # [ 0.004870] CPU features: detected: E0PD worker2 # [ 0.004922] CPU features: detected: RAS Extension Support lb # [ 0.004979] CPU features: detected: RAS Extension Support worker1 # [ 0.004872] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004925] CPU features: detected: RASv1p1 Extension Support lb # [ 0.004982] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.004876] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.004927] CPU features: detected: Random Number Generator lb # [ 0.004985] CPU features: detected: Random Number Generator worker1 # [ 0.004879] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.004930] CPU features: detected: Speculation barrier (SB) lb # [ 0.004987] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.004882] CPU features: detected: Fine Grained Traps worker2 # [ 0.004933] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.004990] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.004886] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.004936] CPU features: detected: TLB range maintenance instructions lb # [ 0.004993] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.004891] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.004940] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.004999] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.004894] CPU features: detected: LSE atomic instructions worker2 # [ 0.004976] alternatives: applying system-wide alternatives lb # [ 0.005036] alternatives: applying system-wide alternatives worker1 # [ 0.004897] CPU features: detected: Privileged Access Never lb # [ 0.008075] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.007952] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.004900] CPU features: detected: PMUv3 client # [ 0.144405] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.004903] CPU features: detected: RAS Extension Support client # [ 0.144422] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.004905] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.008128] Memory: 1406512K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved) lb # [ 0.008231] Memory: 1406544K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved) worker2 # [ 0.008495] devtmpfs: initialized lb # [ 0.008613] devtmpfs: initialized worker1 # [ 0.004908] CPU features: detected: Random Number Generator client # [ 0.144427] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.004911] CPU features: detected: Speculation barrier (SB) client # [ 0.144432] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.010491] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.010463] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.004914] CPU features: detected: Stage-2 Force Write-Back client # [ 0.144436] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.010513] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.010485] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.004917] CPU features: detected: TLB range maintenance instructions client # [ 0.144440] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.010696] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.010681] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.004922] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.144445] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.004958] alternatives: applying system-wide alternatives client # [ 0.144449] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.007902] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.151788] fbcon: Taking over console worker1 # [ 0.008137] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved) client # [ 0.151804] ACPI: Interpreter disabled. worker1 # [ 0.008545] devtmpfs: initialized worker1 # [ 0.010256] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.010279] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.010484] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.010488] 0 pages in range for non-PLT usage worker1 # [ 0.010489] 508288 pages in range for PLT usage worker1 # [ 0.010596] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.010701] 0 pages in range for non-PLT usage lb # [ 0.010685] 0 pages in range for non-PLT usage worker1 # [ 0.011331] DMI not present or invalid. worker2 # [ 0.010702] 508288 pages in range for PLT usage lb # [ 0.010686] 508288 pages in range for PLT usage worker1 # [ 0.014552] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.010810] pinctrl core: initialized pinctrl subsystem lb # [ 0.010804] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.011599] DMI not present or invalid. lb # [ 0.011567] DMI not present or invalid. worker1 # [ 0.016754] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.014825] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.014929] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.016927] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.017045] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.017140] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.017121] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.160854] iommu: Default domain type: Translated worker2 # [ 0.017239] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.017328] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.017143] audit: initializing netlink subsys (disabled) client # [ 0.160864] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.017436] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.017531] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.161252] SCSI subsystem initialized worker1 # [ 0.017717] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.017460] audit: initializing netlink subsys (disabled) lb # [ 0.017552] audit: initializing netlink subsys (disabled) worker1 # [ 0.017718] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.018038] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.018151] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.163200] usbcore: registered new interface driver usbfs worker1 # [ 0.017722] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.018039] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.018154] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.163247] usbcore: registered new interface driver hub worker1 # [ 0.017727] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.018043] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.018158] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.163266] usbcore: registered new device driver usb worker1 # [ 0.017730] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.163537] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.018049] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.018163] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.017754] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.018051] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.018165] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.017762] cpuidle: using governor ladder client # [ 0.163543] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.163554] PTP clock support registered worker2 # [ 0.018075] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.018198] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 client # [ 0.163605] EDAC MC: Ver: 3.0.0 worker2 # [ 0.018083] cpuidle: using governor ladder lb # [ 0.018206] cpuidle: using governor ladder worker2 # [ 0.018089] cpuidle: using governor menu lb # [ 0.018212] cpuidle: using governor menu client # [ 0.168099] scmi_core: SCMI protocol bus registered worker2 # [ 0.018280] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.018394] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. client # [ 0.169086] FPGA manager framework worker2 # [ 0.018294] ASID allocator initialised with 65536 entries lb # [ 0.018409] ASID allocator initialised with 65536 entries client # [ 0.170084] vgaarb: loaded worker2 # [ 0.019437] Serial: AMBA PL011 UART driver lb # [ 0.019590] Serial: AMBA PL011 UART driver client # [ 0.170708] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.024485] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.024891] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.024605] printk: console [ttyAMA0] enabled lb # [ 0.025017] printk: console [ttyAMA0] enabled worker2 # [ 0.143326] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.143558] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.143346] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.143578] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.143350] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.143584] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.017767] cpuidle: using governor menu worker2 # [ 0.143355] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.143588] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.175595] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.017956] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.143359] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.143593] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.175631] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.017971] ASID allocator initialised with 65536 entries worker2 # [ 0.143363] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.143597] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.019138] Serial: AMBA PL011 UART driver worker2 # [ 0.143367] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.143601] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.024272] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.143371] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.143606] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.024421] printk: console [ttyAMA0] enabled client # [ 0.179201] netfs: FS-Cache loaded client # [ 0.179343] pnp: PnP ACPI: disabled worker1 # [ 0.144120] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.144136] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.150627] fbcon: Taking over console lb # [ 0.150894] fbcon: Taking over console worker2 # [ 0.150641] ACPI: Interpreter disabled. lb # [ 0.150908] ACPI: Interpreter disabled. worker1 # [ 0.144141] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.152405] iommu: Default domain type: Translated lb # [ 0.152711] iommu: Default domain type: Translated worker1 # [ 0.144146] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.152414] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.152721] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.144150] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.154069] SCSI subsystem initialized lb # [ 0.154398] SCSI subsystem initialized worker1 # [ 0.144154] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.144158] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.144162] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.183453] NET: Registered PF_INET protocol family worker1 # [ 0.151483] fbcon: Taking over console client # [ 0.183726] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.151499] ACPI: Interpreter disabled. worker1 # [ 0.153386] iommu: Default domain type: Translated worker1 # [ 0.153394] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.155059] SCSI subsystem initialized worker2 # [ 0.161502] usbcore: registered new interface driver usbfs lb # [ 0.162009] usbcore: registered new interface driver usbfs worker2 # [ 0.161534] usbcore: registered new interface driver hub lb # [ 0.162042] usbcore: registered new interface driver hub worker2 # [ 0.161552] usbcore: registered new device driver usb lb # [ 0.162066] usbcore: registered new device driver usb worker2 # [ 0.161828] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.162370] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.161834] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.162376] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.161843] PTP clock support registered lb # [ 0.162386] PTP clock support registered worker2 # [ 0.161890] EDAC MC: Ver: 3.0.0 lb # [ 0.162425] EDAC MC: Ver: 3.0.0 worker2 # [ 0.166611] scmi_core: SCMI protocol bus registered worker1 # [ 0.160405] usbcore: registered new interface driver usbfs lb # [ 0.167120] scmi_core: SCMI protocol bus registered worker1 # [ 0.160436] usbcore: registered new interface driver hub worker2 # [ 0.167551] FPGA manager framework worker1 # [ 0.160454] usbcore: registered new device driver usb lb # [ 0.168084] FPGA manager framework worker2 # [ 0.168534] vgaarb: loaded lb # [ 0.169070] vgaarb: loaded worker1 # [ 0.160711] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.160717] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.169157] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.160726] PTP clock support registered worker2 # [ 0.169748] VFS: Disk quotas dquot_6.6.0 lb # [ 0.169694] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.160773] EDAC MC: Ver: 3.0.0 lb # [ 0.170515] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.169775] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.165386] scmi_core: SCMI protocol bus registered lb # [ 0.170554] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.172128] netfs: FS-Cache loaded worker2 # [ 0.172254] pnp: PnP ACPI: disabled worker1 # [ 0.166345] FPGA manager framework lb # [ 0.172918] netfs: FS-Cache loaded worker1 # [ 0.167265] vgaarb: loaded lb # [ 0.173019] pnp: PnP ACPI: disabled worker1 # [ 0.167896] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.171542] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.171577] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.178136] NET: Registered PF_INET protocol family worker1 # [ 0.175243] netfs: FS-Cache loaded lb # [ 0.178924] NET: Registered PF_INET protocol family worker1 # [ 0.175379] pnp: PnP ACPI: disabled worker2 # [ 0.178408] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.179195] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.214157] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.179124] NET: Registered PF_INET protocol family worker1 # [ 0.179395] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.214206] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.214238] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.214308] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.214450] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.214544] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.214609] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.214647] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.222138] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.222197] NET: Registered PF_XDP protocol family client # [ 0.222217] PCI: CLS 0 bytes, default 64 client # [ 0.222466] Trying to unpack rootfs image as initramfs... client # [ 0.233236] kvm [1]: HYP mode not available lb # [ 0.209422] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.209476] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.209511] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.209599] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.207953] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.208008] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.208037] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.208124] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.208267] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.213722] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.208354] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.209055] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.213820] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.208390] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.209111] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.213857] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.208427] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.213895] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.209145] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.208574] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.214028] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.208597] NET: Registered PF_XDP protocol family worker1 # [ 0.209228] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.208615] PCI: CLS 0 bytes, default 64 lb # [ 0.214051] NET: Registered PF_XDP protocol family lb # [ 0.214069] PCI: CLS 0 bytes, default 64 worker1 # [ 0.209371] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.208880] Trying to unpack rootfs image as initramfs... lb # [ 0.214349] Trying to unpack rootfs image as initramfs... worker1 # [ 0.209455] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.209491] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.209527] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.209633] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.227295] kvm [1]: HYP mode not available worker1 # [ 0.209675] NET: Registered PF_XDP protocol family worker1 # [ 0.209694] PCI: CLS 0 bytes, default 64 worker1 # [ 0.209952] Trying to unpack rootfs image as initramfs... worker2 # [ 0.226736] kvm [1]: HYP mode not available worker1 # [ 0.225483] kvm [1]: HYP mode not available client # [ 0.331304] Initialise system trusted keyrings client # [ 0.332052] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.333347] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.334109] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.323582] Initialise system trusted keyrings worker2 # [ 0.324311] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.326802] Initialise system trusted keyrings worker1 # [ 0.321239] Initialise system trusted keyrings lb # [ 0.327526] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.322027] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.323279] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.325607] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.362845] Key type asymmetric registered worker2 # [ 0.325679] 9p: Installing v9fs 9p2000 file system support lb # [ 0.334198] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.362874] Asymmetric key parser 'x509' registered client # [ 0.362955] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.365093] io scheduler mq-deadline registered client # [ 0.365104] io scheduler kyber registered lb # [ 0.334988] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.324079] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.345601] Key type asymmetric registered worker2 # [ 0.345626] Asymmetric key parser 'x509' registered client # [ 0.374836] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.345711] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.376267] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.344036] Key type asymmetric registered client # [ 0.376648] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.344061] Asymmetric key parser 'x509' registered client # [ 0.376666] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.344126] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.376678] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.353202] io scheduler mq-deadline registered worker2 # [ 0.353222] io scheduler kyber registered client # [ 0.376687] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.376706] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.355671] Key type asymmetric registered client # [ 0.376731] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.355698] Asymmetric key parser 'x509' registered client # [ 0.376808] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.355771] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.376817] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.357960] io scheduler mq-deadline registered worker1 # [ 0.351954] io scheduler mq-deadline registered client # [ 0.376823] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.357970] io scheduler kyber registered worker1 # [ 0.351972] io scheduler kyber registered client # [ 0.376829] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.376834] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.376933] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.377371] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.361303] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.377555] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.377571] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.377600] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.377616] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.362653] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.378061] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.363038] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.378241] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.360048] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.363054] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.378256] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.378285] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.363066] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.363075] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.361360] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.363093] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.361758] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.363117] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.361775] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.363193] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.363202] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.361788] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.363208] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.361796] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.363213] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.361817] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.363218] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.361840] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.363273] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.361913] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.363708] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.361923] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.363900] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.361929] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.363916] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.361934] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.363946] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.363961] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.361939] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.361996] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.364414] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.373850] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.402754] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.364597] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.374545] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.362429] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.402953] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.364613] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.362616] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.402969] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.374909] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.364642] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.362633] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.402999] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.374926] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.362662] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.365093] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint client # [ 0.403503] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker1 # [ 0.362678] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.374958] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.403687] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.363144] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.403703] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.374967] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.363329] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.403736] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.374989] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.363344] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.404198] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker1 # [ 0.363373] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.375022] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.404384] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.375101] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.363823] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint client # [ 0.404400] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.375110] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.404429] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.375116] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.404886] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.375121] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.405072] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.375127] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.405087] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.375245] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.405117] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.375687] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.405572] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.375872] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.405757] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.375888] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.405773] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.375917] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.405802] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.375933] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.405818] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.376391] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.406301] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.376575] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.406492] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.376590] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.406521] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.376620] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.406990] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.393350] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.377069] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.393382] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.407179] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.377251] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.393412] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.407215] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.377267] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.393922] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint client # [ 0.407610] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.377297] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.394127] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.407795] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.377776] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.394142] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.408039] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.377958] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.394174] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392092] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f] client # [ 0.408332] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.377974] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.408350] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.392122] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.394639] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint lb # [ 0.378006] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.394827] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] client # [ 0.408379] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392152] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.378476] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.394842] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.392655] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint lb # [ 0.378659] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] client # [ 0.408852] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.394872] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392843] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.409037] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.378675] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409052] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.395335] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.392859] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.378705] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.395525] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] client # [ 0.409082] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392891] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.379174] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.395541] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409658] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.379355] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.393351] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint worker2 # [ 0.395570] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409669] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.379371] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.393536] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.393551] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.396023] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.379401] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409675] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.396206] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.393580] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.379847] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.396221] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409720] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.394067] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.380029] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.396250] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409767] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.394251] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.380044] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.396266] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.409813] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.394266] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.380074] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.396764] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.394295] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.380089] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.409859] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.396951] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.394746] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.380572] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.409905] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.396980] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.394932] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.380759] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409953] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.397445] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.394947] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.380789] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.397632] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.394976] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409999] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.381243] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.397661] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.394992] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.410046] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.381429] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.395453] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.398059] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.381458] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.410093] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.398248] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.395642] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.410164] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.395672] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.398491] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.410211] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.398786] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.396154] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.410232] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.398803] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.396342] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.410254] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.398832] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.396371] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.410275] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.399294] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.396757] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.410299] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.399476] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.396941] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.399491] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.410320] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.397190] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.399521] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.410342] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.397466] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.400112] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.410364] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.397483] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.400122] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.410386] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.397513] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.410408] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.400127] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.397983] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.410430] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.400172] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.398168] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.410453] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.398183] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.400219] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.410474] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.398212] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400265] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.410496] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.398791] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.410517] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned worker2 # [ 0.400311] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.398802] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.410538] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.400356] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.398807] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.410559] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.429938] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.400402] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.410581] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned lb # [ 0.430140] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.398852] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.400447] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.410602] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned worker1 # [ 0.398900] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.430394] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.410624] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned worker2 # [ 0.400493] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.430672] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.398946] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.410646] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned lb # [ 0.430690] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.400539] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.410672] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.398993] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.430719] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.410682] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.400611] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.399039] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.431172] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.410686] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.400657] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.399090] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.431352] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.400679] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.431368] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.399137] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.400700] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.431398] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400722] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.399183] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.431971] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.400745] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.399231] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.431982] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.400767] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.399301] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.431988] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.400788] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.399348] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.400810] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.432032] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.399370] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.400831] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.432079] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.399392] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.400853] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.432124] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.475932] pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.399413] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.400875] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.399437] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.432171] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.400898] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.399459] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.432218] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.400919] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.399482] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.400940] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.432265] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.400961] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned worker1 # [ 0.399504] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.432312] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.400982] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.399526] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.432360] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.401003] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.399548] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.401024] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned lb # [ 0.432406] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.399570] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.401045] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned lb # [ 0.432465] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.399593] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.401067] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned worker1 # [ 0.399615] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.432510] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.401088] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned worker1 # [ 0.399636] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.432531] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.401115] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.399658] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned lb # [ 0.432552] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.401125] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.399679] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.401130] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.432573] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.399701] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.432597] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.399722] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned lb # [ 0.432619] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.399743] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned lb # [ 0.432641] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.399766] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned lb # [ 0.432662] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.399787] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned lb # [ 0.432684] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.399814] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.399823] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.432706] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.399828] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.432728] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.432750] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.432770] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.432791] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.432812] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned lb # [ 0.432833] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.432854] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.466353] pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.432875] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned lb # [ 0.432896] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned lb # [ 0.432918] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned lb # [ 0.432939] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned lb # [ 0.432966] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.432975] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.465053] pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.432981] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.501800] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.434160] pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.508015] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.510114] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.486914] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.489056] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.520209] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.493115] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.485778] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.522152] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) worker2 # [ 0.498946] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.500921] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.533684] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) lb # [ 0.503141] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.495980] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.499413] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.536734] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) worker1 # [ 0.503124] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.512127] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) lb # [ 0.507326] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.509376] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.511471] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) lb # [ 0.513654] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) worker1 # [ 0.509224] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) worker2 # [ 0.518494] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) worker2 # [ 0.520771] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) client # [ 0.546874] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.549226] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.551103] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.553512] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.520477] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) worker1 # [ 0.522710] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) worker2 # [ 0.530652] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.532942] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.535283] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) lb # [ 0.537673] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.533832] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.541642] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.572028] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.544238] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.536876] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.538772] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.544366] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.574765] msm_serial: driver initialized worker2 # [ 0.547417] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.574918] SuperH (H)SCI(F) driver initialized client # [ 0.574973] STM32 USART driver initialized lb # [ 0.555116] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.549331] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.560353] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.562945] msm_serial: driver initialized worker1 # [ 0.558437] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.564384] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.563086] SuperH (H)SCI(F) driver initialized worker2 # [ 0.563138] STM32 USART driver initialized lb # [ 0.571050] msm_serial: driver initialized worker1 # [ 0.565148] msm_serial: driver initialized lb # [ 0.571203] SuperH (H)SCI(F) driver initialized worker1 # [ 0.565305] SuperH (H)SCI(F) driver initialized lb # [ 0.571261] STM32 USART driver initialized worker1 # [ 0.565356] STM32 USART driver initialized client # [ 0.609594] loop: module loaded client # [ 0.609786] virtio_blk virtio5: 1/0/0 default/read/poll queues client # [ 0.610535] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.619448] megasas: 07.734.00.00-rc1 client # [ 0.620191] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.597799] loop: module loaded client # [ 0.622087] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.598030] virtio_blk virtio5: 1/0/0 default/read/poll queues client # [ 0.622113] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.623765] Using buffer write method worker2 # [ 0.598779] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.594857] loop: module loaded client # [ 0.623825] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.595051] virtio_blk virtio5: 1/0/0 default/read/poll queues worker1 # [ 0.595815] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.625481] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.625510] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.609953] loop: module loaded lb # [ 0.610179] virtio_blk virtio5: 1/0/0 default/read/poll queues worker2 # [ 0.605824] megasas: 07.734.00.00-rc1 lb # [ 0.611006] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.606618] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.643801] Using buffer write method client # [ 0.643840] Concatenating MTD devices: client # [ 0.643844] (0): "0.flash" worker2 # [ 0.608789] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.643848] (1): "0.flash" client # [ 0.643852] into device "0.flash" worker2 # [ 0.608813] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.610423] Using buffer write method worker2 # [ 0.610480] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.612134] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.612186] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.615198] megasas: 07.734.00.00-rc1 worker1 # [ 0.608594] megasas: 07.734.00.00-rc1 lb # [ 0.615978] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.609396] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.619945] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.611376] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.619971] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.621677] Using buffer write method worker1 # [ 0.611400] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.613107] Using buffer write method worker2 # [ 0.625760] Using buffer write method worker2 # [ 0.625794] Concatenating MTD devices: worker2 # [ 0.625798] (0): "0.flash" worker2 # [ 0.625802] (1): "0.flash" worker1 # [ 0.613171] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.625805] into device "0.flash" worker1 # [ 0.615032] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.615093] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.629791] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.628878] Using buffer write method lb # [ 0.631778] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.628912] Concatenating MTD devices: worker1 # [ 0.628916] (0): "0.flash" lb # [ 0.631812] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.628920] (1): "0.flash" lb # [ 0.633673] Using buffer write method worker1 # [ 0.628923] into device "0.flash" lb # [ 0.641748] Concatenating MTD devices: lb # [ 0.641755] (0): "0.flash" lb # [ 0.641760] (1): "0.flash" lb # [ 0.641764] into device "0.flash" client # [ 0.886742] Freeing initrd memory: 26160K client # [ 0.892967] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.896775] thunder_xcv, ver 1.0 client # [ 0.896821] thunder_bgx, ver 1.0 client # [ 0.896855] nicpf, ver 1.0 client # [ 0.897418] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.897425] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.897451] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.897460] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.897492] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.897498] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.897522] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.897527] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.897664] sky2: driver version 1.30 client # [ 0.906495] usbcore: registered new interface driver usb-storage client # [ 0.906573] usbcore: registered new interface driver usbserial_generic client # [ 0.906589] usbserial: USB Serial support registered for generic worker2 # [ 0.880288] Freeing initrd memory: 26148K client # [ 0.907490] ehci-pci 0000:00:0a.0: EHCI Host Controller client # [ 0.907517] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 client # [ 0.907795] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 client # [ 0.912900] hv_vmbus: registering driver hyperv_keyboard client # [ 0.914491] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.886233] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.914519] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:49:10 UTC (1790074150) client # [ 0.916571] i2c_dev: i2c /dev entries driver worker1 # [ 0.882004] Freeing initrd memory: 26148K client # [ 0.919887] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.889956] thunder_xcv, ver 1.0 client # [ 0.919898] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.889998] thunder_bgx, ver 1.0 worker2 # [ 0.890021] nicpf, ver 1.0 client # [ 0.920171] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.890551] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.920566] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.889697] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.890558] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.922312] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.890581] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.922590] usbcore: registered new interface driver usbhid client # [ 0.922597] usbhid: USB HID core driver worker2 # [ 0.890590] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.890617] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.890622] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.927323] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 client # [ 0.927636] hub 1-0:1.0: USB hub found worker2 # [ 0.890644] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.927659] hub 1-0:1.0: 6 ports detected worker2 # [ 0.890649] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.890778] sky2: driver version 1.30 lb # [ 0.901280] Freeing initrd memory: 26140K worker2 # [ 0.892282] usbcore: registered new interface driver usb-storage worker1 # [ 0.894183] thunder_xcv, ver 1.0 worker2 # [ 0.892347] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.894251] thunder_bgx, ver 1.0 worker1 # [ 0.894285] nicpf, ver 1.0 worker2 # [ 0.892362] usbserial: USB Serial support registered for generic worker1 # [ 0.895069] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.892929] hv_vmbus: registering driver hyperv_keyboard client # [ 0.930732] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.895078] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.893911] ehci-pci 0000:00:0a.0: EHCI Host Controller client # [ 0.932318] drop_monitor: Initializing network drop monitor service worker1 # [ 0.895114] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.893989] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 client # [ 0.932446] NET: Registered PF_INET6 protocol family worker1 # [ 0.895123] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.894243] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 worker1 # [ 0.895167] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.935382] Segment Routing with IPv6 client # [ 0.935405] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.895174] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.906182] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 client # [ 0.935434] NET: Registered PF_PACKET protocol family lb # [ 0.907297] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.906546] hub 1-0:1.0: USB hub found worker1 # [ 0.895210] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.906565] hub 1-0:1.0: 6 ports detected worker1 # [ 0.895217] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.895411] sky2: driver version 1.30 worker2 # [ 0.909140] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.937086] 9pnet: Installing 9P2000 support worker2 # [ 0.909844] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:49:10 UTC (1790074150) worker2 # [ 0.910163] i2c_dev: i2c /dev entries driver worker1 # [ 0.904748] usbcore: registered new interface driver usb-storage worker1 # [ 0.904845] usbcore: registered new interface driver usbserial_generic client # [ 0.939884] Key type dns_resolver registered worker1 # [ 0.904859] usbserial: USB Serial support registered for generic lb # [ 0.911118] thunder_xcv, ver 1.0 worker1 # [ 0.905555] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.911163] thunder_bgx, ver 1.0 lb # [ 0.911186] nicpf, ver 1.0 lb # [ 0.911767] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.911774] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.909955] ehci-pci 0000:00:0a.0: EHCI Host Controller lb # [ 0.911798] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.914398] sdhci: Secure Digital Host Controller Interface driver client # [ 0.946357] registered taskstats version 1 worker1 # [ 0.909993] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 lb # [ 0.911806] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.914409] sdhci: Copyright(c) Pierre Ossman client # [ 0.946519] Loading compiled-in X.509 certificates worker1 # [ 0.910222] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 lb # [ 0.911835] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.914661] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.911840] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.915019] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.911863] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.916705] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.912887] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.911869] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.916943] usbcore: registered new interface driver usbhid lb # [ 0.912002] sky2: driver version 1.30 worker2 # [ 0.916950] usbhid: USB HID core driver worker1 # [ 0.912919] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:49:10 UTC (1790074150) lb # [ 0.913600] usbcore: registered new interface driver usb-storage worker1 # [ 0.913284] i2c_dev: i2c /dev entries driver lb # [ 0.913648] usbcore: registered new interface driver usbserial_generic lb # [ 0.913662] usbserial: USB Serial support registered for generic lb # [ 0.914473] ehci-pci 0000:00:0a.0: EHCI Host Controller worker2 # [ 0.922388] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.914501] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 lb # [ 0.914752] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 worker2 # [ 0.923862] drop_monitor: Initializing network drop monitor service worker2 # [ 0.923998] NET: Registered PF_INET6 protocol family client # [ 0.955265] Demotion targets for Node 0: null lb # [ 0.926251] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 worker1 # [ 0.918402] sdhci: Secure Digital Host Controller Interface driver client # [ 0.955378] Key type .fscrypt registered worker2 # [ 0.926997] Segment Routing with IPv6 lb # [ 0.926620] hub 1-0:1.0: USB hub found worker1 # [ 0.918413] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.927019] In-situ OAM (IOAM) with IPv6 client # [ 0.955384] Key type fscrypt-provisioning registered lb # [ 0.926636] hub 1-0:1.0: 6 ports detected worker2 # [ 0.927047] NET: Registered PF_PACKET protocol family worker1 # [ 0.918694] Synopsys Designware Multimedia Card Interface Driver client # [ 0.955484] ima: No TPM chip found, activating TPM-bypass! client # [ 0.955503] ima: Allocated hash algorithm: sha1 worker1 # [ 0.919122] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.955523] ima: No architecture policies found worker2 # [ 0.928579] 9pnet: Installing 9P2000 support worker1 # [ 0.919949] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 lb # [ 0.929200] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.920269] hub 1-0:1.0: USB hub found worker1 # [ 0.920291] hub 1-0:1.0: 6 ports detected client # [ 0.959757] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.930722] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.930749] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:49:10 UTC (1790074150) worker2 # [ 0.931312] Key type dns_resolver registered lb # [ 0.931054] i2c_dev: i2c /dev entries driver worker1 # [ 0.926087] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.926347] usbcore: registered new interface driver usbhid worker1 # [ 0.926355] usbhid: USB HID core driver lb # [ 0.935940] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.929322] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.935950] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.930920] drop_monitor: Initializing network drop monitor service lb # [ 0.936221] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.931113] NET: Registered PF_INET6 protocol family lb # [ 0.936599] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.934166] Segment Routing with IPv6 worker1 # [ 0.934185] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.934215] NET: Registered PF_PACKET protocol family worker2 # [ 0.937528] registered taskstats version 1 lb # [ 0.940968] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.937670] Loading compiled-in X.509 certificates lb # [ 0.941213] usbcore: registered new interface driver usbhid lb # [ 0.941220] usbhid: USB HID core driver lb # [ 0.944122] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.946235] Demotion targets for Node 0: null lb # [ 0.945632] drop_monitor: Initializing network drop monitor service worker2 # [ 0.946340] Key type .fscrypt registered lb # [ 0.947644] NET: Registered PF_INET6 protocol family worker2 # [ 0.946347] Key type fscrypt-provisioning registered worker2 # [ 0.946445] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.948844] Segment Routing with IPv6 worker2 # [ 0.946464] ima: Allocated hash algorithm: sha1 lb # [ 0.948863] In-situ OAM (IOAM) with IPv6 client # [ 0.978080] clk: Disabling unused clocks worker2 # [ 0.946485] ima: No architecture policies found worker1 # [ 0.935933] 9pnet: Installing 9P2000 support lb # [ 0.948892] NET: Registered PF_PACKET protocol family client # [ 0.978105] PM: genpd: Disabling unused power domains worker1 # [ 0.938211] Key type dns_resolver registered worker2 # [ 0.950483] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.982400] Freeing unused kernel memory: 4736K lb # [ 0.950700] 9pnet: Installing 9P2000 support client # [ 0.982598] Run /init as init process lb # [ 0.952875] Key type dns_resolver registered worker1 # [ 0.944973] registered taskstats version 1 worker1 # [ 0.945138] Loading compiled-in X.509 certificates worker1 # [ 0.953894] Demotion targets for Node 0: null worker1 # [ 0.954014] Key type .fscrypt registered worker1 # [ 0.954021] Key type fscrypt-provisioning registered lb # [ 0.959702] registered taskstats version 1 worker1 # [ 0.954126] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.959878] Loading compiled-in X.509 certificates worker1 # [ 0.954146] ima: Allocated hash algorithm: sha1 worker1 # [ 0.954173] ima: No architecture policies found worker1 # [ 0.958245] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.997014] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.968322] Demotion targets for Node 0: null lb # [ 0.968433] Key type .fscrypt registered worker2 # [ 0.968364] clk: Disabling unused clocks lb # [ 0.968440] Key type fscrypt-provisioning registered worker2 # [ 0.968398] PM: genpd: Disabling unused power domains lb # [ 0.968534] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.968554] ima: Allocated hash algorithm: sha1 lb # [ 0.968576] ima: No architecture policies found lb # [ 0.972639] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.972685] Freeing unused kernel memory: 4736K worker2 # [ 0.972884] Run /init as init process worker1 # [ 0.975733] clk: Disabling unused clocks worker1 # [ 0.975760] PM: genpd: Disabling unused power domains worker1 # [ 0.979960] Freeing unused kernel memory: 4736K worker1 # [ 0.980162] Run /init as init process worker2 # [ 0.987343] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.991302] clk: Disabling unused clocks lb # [ 0.991326] PM: genpd: Disabling unused power domains lb # [ 0.995568] Freeing unused kernel memory: 4736K lb # [ 0.995780] Run /init as init process worker1 # [ 0.994191] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.011961] systemd[1]: Successfully made /usr/ read-only. client # [ 1.178786] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.153243] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.173777] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.168015] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.330443] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.302820] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.334605] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.342843] systemd[1]: Detected virtualization qemu. client # [ 1.344154] systemd[1]: Detected architecture arm64. client # [ 1.345514] systemd[1]: Running in initrd. client # [ 1.346158] systemd[1]: Initializing machine ID from random generator. client # [ 1.346342] systemd[1]: Hostname set to . worker2 # [ 1.321941] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.317851] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.326331] systemd[1]: Detected virtualization qemu. worker2 # [ 1.327045] systemd[1]: Detected architecture arm64. worker2 # [ 1.327726] systemd[1]: Running in initrd. lb # [ 1.327506] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.328782] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.329966] systemd[1]: Hostname set to . worker1 # [ 1.329053] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.341546] systemd[1]: Detected virtualization qemu. worker1 # [ 1.343776] systemd[1]: Detected architecture arm64. worker1 # [ 1.345930] systemd[1]: Running in initrd. worker1 # [ 1.348686] systemd[1]: Initializing machine ID from random generator. lb # [ 1.346943] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.351840] systemd[1]: Hostname set to . lb # [ 1.358714] systemd[1]: Detected virtualization qemu. lb # [ 1.360524] systemd[1]: Detected architecture arm64. lb # [ 1.362385] systemd[1]: Running in initrd. lb # [ 1.364837] systemd[1]: Initializing machine ID from random generator. lb # [ 1.367528] systemd[1]: Hostname set to . client # [ 1.410909] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 worker2 # [ 1.381440] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 lb # [ 1.410011] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 worker1 # [ 1.408180] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 worker2 # [ 1.501208] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.530831] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.533757] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.528010] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.621715] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.612802] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.647744] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.659114] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.689567] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.689761] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 worker2 # [ 1.663334] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 client # [ 1.711387] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.682141] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.721317] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.684872] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 client # [ 1.722643] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.724359] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.724403] systemd[1]: Reached target Path Units. client # [ 1.724426] systemd[1]: Reached target Slice Units. client # [ 1.724449] systemd[1]: Reached target Swaps. client # [ 1.724472] systemd[1]: Reached target Timer Units. lb # [ 1.697395] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.724688] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.724862] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.725027] systemd[1]: Listening on Journal Sockets. client # [ 1.725177] systemd[1]: Listening on udev Control Socket. client # [ 1.725318] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.703259] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 client # [ 1.725343] systemd[1]: Reached target Socket Units. client # [ 1.737713] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.708085] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.746920] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker2 # [ 1.718164] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.748127] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.753716] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.727691] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.728970] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.730475] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.731581] systemd[1]: Reached target Path Units. worker2 # [ 1.732428] systemd[1]: Reached target Slice Units. worker2 # [ 1.733332] systemd[1]: Reached target Swaps. worker2 # [ 1.734174] systemd[1]: Reached target Timer Units. worker2 # [ 1.735233] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.729807] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.736532] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.737763] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.734086] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.737911] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.738054] systemd[1]: Listening on udev Kernel Socket. client # [ 1.769643] systemd[1]: Starting Journal Service... worker1 # [ 1.735186] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.738078] systemd[1]: Reached target Socket Units. worker1 # [ 1.736455] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.737422] systemd[1]: Reached target Path Units. worker2 # [ 1.743369] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.738250] systemd[1]: Reached target Slice Units. worker1 # [ 1.738991] systemd[1]: Reached target Swaps. worker1 # [ 1.739656] systemd[1]: Reached target Timer Units. worker1 # [ 1.740680] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.741796] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.742831] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.743735] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.744744] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.750573] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 1.745536] systemd[1]: Reached target Socket Units. worker2 # [ 1.751785] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.747846] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.761368] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.790917] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.756135] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 1.791055] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.756227] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.769350] systemd[1]: Starting Journal Service... worker1 # [ 1.764235] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.806906] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.784232] systemd[1]: Starting Journal Service... lb # [ 1.798055] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.795663] systemd[1]: Starting Load Kernel Modules... client # [ 1.831713] systemd-journald[73]: Collecting audit messages is disabled. worker1 # [ 1.796587] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.808110] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.809232] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.838910] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.809476] systemd[1]: Starting Load Kernel Modules... lb # [ 1.810552] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.811551] systemd[1]: Reached target Path Units. worker2 # [ 1.809605] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.840769] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 1.812304] systemd[1]: Reached target Slice Units. lb # [ 1.813073] systemd[1]: Reached target Swaps. lb # [ 1.813789] systemd[1]: Reached target Timer Units. lb # [ 1.814763] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.815912] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.816968] systemd[1]: Listening on Journal Sockets. lb # [ 1.818030] systemd[1]: Listening on udev Control Socket. lb # [ 1.818994] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.817628] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.819828] systemd[1]: Reached target Socket Units. worker1 # [ 1.812332] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.822293] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.851156] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 1.855098] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.829891] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker2 # [ 1.830482] systemd-journald[73]: Collecting audit messages is disabled. worker2 # [ 1.831562] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.830995] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.824171] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.825144] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker2 # [ 1.834137] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 1.825433] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 1.825659] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.837373] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.838328] systemd-journald[73]: Collecting audit messages is disabled. worker2 # [ 1.845665] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker2 # [ 1.846081] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.878836] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.881055] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.852086] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.855644] systemd[1]: Starting Journal Service... worker1 # [ 1.852025] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.854102] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.857793] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.865286] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.895162] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.864713] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.902622] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 worker2 # [ 1.873340] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 worker2 # [ 1.873587] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.873599] [drm] features: -context_init worker2 # [ 1.874330] [drm] number of scanouts: 1 worker2 # [ 1.874348] [drm] number of cap sets: 0 lb # [ 1.878273] systemd[1]: Starting Load Kernel Modules... lb # [ 1.882060] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.876113] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 worker1 # [ 1.876354] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.876363] [drm] features: -context_init worker1 # [ 1.877098] [drm] number of scanouts: 1 worker1 # [ 1.877116] [drm] number of cap sets: 0 client # [ 1.919000] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.919022] [drm] features: -context_init client # [ 1.919826] [drm] number of scanouts: 1 client # [ 1.919845] [drm] number of cap sets: 0 lb # [ 1.893967] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.893892] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.893939] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 worker1 # [ 1.893141] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.895670] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.895693] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 client # [ 1.927061] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic client # [ 1.927080] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 worker1 # [ 1.904398] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.917870] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.947479] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.920575] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.950081] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.918553] systemd-journald[73]: Collecting audit messages is disabled. lb # [ 1.922851] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.924214] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 1.924724] systemd[1]: Finished Load Kernel Module 9pnet_virtio. lb # [ 1.925183] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.955143] Console: switching to colour frame buffer device 160x50 client # [ 1.962205] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 1.924612] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.933480] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.940467] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 1.943664] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.956360] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.991486] systemd[1]: Finished Load Kernel Modules. client # [ 1.994846] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 1.966073] systemd[1]: Finished Load Kernel Modules. worker1 # [ 1.960598] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.960981] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.969284] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.961019] systemd[1]: Reached target Local File Systems. worker1 # [ 1.969038] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.005994] systemd[1]: Started Journal Service. lb # [ 1.971646] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 2.004950] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 2.008221] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 1.993803] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 2.009361] systemd-modules-load[74]: Module 'virtio_console' is built in client # [ 2.010436] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 1.988097] systemd[1]: Finished Load Kernel Modules. client # [ 2.011400] systemd-modules-load[74]: Module 'virtio_rng' is built in lb # [ 1.998426] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.992263] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 1.997528] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.998003] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.998041] systemd[1]: Reached target Local File Systems. worker2 # [ 2.005609] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.008683] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 client # [ 2.024550] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.008938] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 2.008951] [drm] features: -context_init client # [ 2.025684] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 2.009653] [drm] number of scanouts: 1 lb # [ 2.009673] [drm] number of cap sets: 0 client # [ 2.026747] systemd[1]: Reached target Local File Systems. client # [ 2.027641] systemd[1]: Starting Create System Files and Directories... lb # [ 2.013317] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 2.041420] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.045048] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.008508] systemd-modules-load[74]: Inserted module 'dm_mod' worker1 # [ 2.025548] systemd[1]: Started Journal Service. worker1 # [ 2.013039] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker1 # [ 2.014182] systemd-modules-load[74]: Module 'virtio_console' is built in worker1 # [ 2.015227] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker2 # [ 2.037457] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.034688] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic worker2 # [ 2.039873] systemd[1]: Started Journal Service. lb # [ 2.034722] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 worker2 # [ 2.024989] systemd-modules-load[75]: Inserted module 'dm_mod' worker2 # [ 2.033186] systemd-modules-load[75]: Module 'virtio_balloon' is built in worker2 # [ 2.034308] systemd-modules-load[75]: Module 'virtio_console' is built in worker2 # [ 2.035361] systemd-modules-load[75]: Inserted module 'virtio_gpu' worker1 # [ 2.032450] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.033683] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.048436] systemd-modules-load[75]: Module 'virtio_rng' is built in worker2 # [ 2.049491] systemd[1]: Starting Create System Files and Directories... lb # [ 2.058606] Console: switching to colour frame buffer device 160x50 worker1 # [ 2.045941] systemd[1]: Finished Apply Kernel Variables. client # [ 2.084947] systemd[1]: Finished Create System Files and Directories. lb # [ 2.078480] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.078868] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 2.078893] systemd[1]: Reached target Local File Systems. lb # [ 2.086124] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 2.069825] systemd-udevd[80]: Using default interface naming scheme 'v261'. lb # [ 2.094387] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 2.109959] systemd-udevd[81]: Using default interface naming scheme 'v261'. worker1 # [ 2.084553] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.093703] systemd-udevd[81]: Using default interface naming scheme 'v261'. lb # [ 2.104579] systemd-modules-load[75]: Inserted module 'dm_mod' worker2 # [ 2.105564] systemd[1]: Finished Create System Files and Directories. lb # [ 2.119515] systemd[1]: Finished Load Kernel Modules. lb # [ 2.121617] systemd[1]: Started Journal Service. client # [ 2.138310] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.110969] systemd-modules-load[75]: Module 'virtio_balloon' is built in lb # [ 2.112257] systemd-modules-load[75]: Module 'virtio_console' is built in worker1 # [ 2.105977] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.115638] systemd-modules-load[75]: Inserted module 'virtio_gpu' worker2 # [ 2.126519] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.128472] systemd-modules-load[75]: Module 'virtio_rng' is built in lb # [ 2.129549] systemd[1]: Starting Apply Kernel Variables... lb # [ 2.144223] systemd[1]: Starting Create System Files and Directories... lb # [ 2.174095] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 2.204170] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.168082] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.184095] systemd[1]: Starting Virtual Console Setup... lb # [ 2.192924] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.201858] systemd[1]: Finished Create System Files and Directories. lb # [ 2.218718] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.248521] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.252211] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.220530] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.224071] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.236537] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.239944] systemd[1]: Finished Virtual Console Setup. lb # [ 2.278929] systemd[1]: Starting Virtual Console Setup... lb # [ 2.336517] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.339789] systemd[1]: Finished Virtual Console Setup. client # [ 2.825589] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.826565] systemd[1]: Reached target System Initialization. client # [ 2.828155] systemd[1]: Reached target Basic System. worker2 # [ 2.814350] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.815921] systemd[1]: Reached target System Initialization. worker2 # [ 2.820415] systemd[1]: Reached target Basic System. worker1 # [ 2.825439] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.826500] systemd[1]: Reached target System Initialization. worker1 # [ 2.827318] systemd[1]: Reached target Basic System. client # [ 2.956193] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.929367] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.930459] systemd[1]: Reached target System Initialization. lb # [ 2.931400] systemd[1]: Reached target Basic System. client # [ 2.998587] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.004267] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.977614] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.981260] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.995421] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 2.992182] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.002159] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.998441] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.064575] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.067106] systemd[1]: Reached target Initrd Root Device. client # [ 3.070969] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.061267] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.063720] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.066152] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.079746] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.082444] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.085447] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.121886] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.096491] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.099073] (udev-worker)[106]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.130157] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.104294] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.139887] systemd[1]: Mounting /sysroot... worker2 # [ 3.127481] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.122484] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.136157] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.127620] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.141039] systemd[1]: Mounting /sysroot... worker1 # [ 3.139128] systemd[1]: Mounting /sysroot... client # [ 3.189991] EXT4-fs (vda): mounted filesystem d4dd8969-73c0-43cf-ab04-047d2883e63d r/w with ordered data mode. Quota mode: none. client # [ 3.181397] systemd[1]: Mounted /sysroot. client # [ 3.182387] systemd[1]: Reached target Initrd Root File System. client # [ 3.188131] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.186317] EXT4-fs (vda): mounted filesystem 47eff782-01e5-4cab-8c65-8fec002074a1 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.179682] systemd[1]: Mounted /sysroot. lb # [ 3.180626] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.181239] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.187805] EXT4-fs (vda): mounted filesystem bf9ef386-e4ac-4152-8954-4575120e86c0 r/w with ordered data mode. Quota mode: none. worker1 # [ 3.177472] systemd[1]: Mounted /sysroot. worker2 # [ 3.186423] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.187319] systemd[1]: Reached target Initrd Root Device. client # [ 3.216231] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.180810] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.185211] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.193751] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.222383] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.226991] systemd[1]: Reloading... worker2 # [ 3.211922] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.220092] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.211707] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.223966] systemd[1]: Reloading... worker1 # [ 3.218390] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.222800] systemd[1]: Reloading... lb # [ 3.251850] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.259236] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.265365] systemd[1]: Mounting /sysroot... lb # [ 3.314424] EXT4-fs (vda): mounted filesystem 263eff33-f374-436f-b58d-fd57b4da33c1 r/w with ordered data mode. Quota mode: none. lb # [ 3.306944] systemd[1]: Mounted /sysroot. lb # [ 3.309346] systemd[1]: Reached target Initrd Root File System. lb # [ 3.314982] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.344316] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.351557] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.355223] systemd[1]: Reloading... client # [ 3.439287] systemd[1]: Reloading finished in 214 ms. worker2 # [ 3.433666] systemd[1]: Reloading finished in 211 ms. worker1 # [ 3.436102] systemd[1]: Reloading finished in 210 ms. client # [ 3.475268] systemd-sysroot-fstab-check[120]: Requesting initrd-fs.target/start/replace... client # [ 3.478493] systemd-sysroot-fstab-check[120]: Requesting swap.target/start/replace... client # [ 3.483795] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 3.486311] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.462211] systemd-sysroot-fstab-check[120]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.468307] systemd-sysroot-fstab-check[120]: Requesting swap.target/start/replace... client # [ 3.496365] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.497606] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.470771] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 3.470673] systemd-sysroot-fstab-check[120]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.480769] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.482753] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.474997] systemd-sysroot-fstab-check[120]: Requesting swap.target/start/replace... worker2 # [ 3.485232] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.480944] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 3.517981] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 3.483257] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.519826] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 3.492474] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.503686] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 3.495259] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.506700] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 3.518968] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 3.520615] systemd[1]: Finished Load Kernel Module 9pnet_virtio. lb # [ 3.566870] systemd[1]: Reloading finished in 211 ms. lb # [ 3.598147] systemd-sysroot-fstab-check[120]: Requesting initrd-fs.target/start/replace... lb # [ 3.605471] systemd-sysroot-fstab-check[120]: Requesting swap.target/start/replace... lb # [ 3.607415] systemd[1]: Starting Load Kernel Module 9pnet_virtio... lb # [ 3.614724] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.618079] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.620742] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.641400] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 3.643040] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 3.799898] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.812085] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.822143] systemd[1]: Mounting /sysroot/run... client # [ 3.831977] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.809266] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.800900] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.826984] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.820135] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.832195] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.825255] systemd[1]: Mounting /sysroot/run... client # [ 3.875197] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.841589] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.852766] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.899764] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 3.874919] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.866866] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.909611] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.920462] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.898720] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.894204] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.931386] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.905421] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.918311] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.947920] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.914500] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.924081] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.952326] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.918760] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.928673] systemd[1]: Mounting /sysroot/run... worker2 # [ 3.936551] systemd[1]: Mounted /sysroot/run. lb # [ 3.945315] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.949346] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.978328] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.942740] systemd[1]: Mounted /sysroot/run. client # [ 3.979935] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.952971] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.954765] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.955614] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 3.957605] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.964667] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.976178] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.967615] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.979221] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.981239] systemd[1]: Mounting /sysroot/nix/store... lb # [ 3.992057] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.995141] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.003201] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 4.021745] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.025985] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.027176] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.030602] systemd[1]: Starting Find NixOS closure... lb # [ 4.031852] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.041022] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.048926] systemd[1]: Mounted /sysroot/run. lb # [ 4.066931] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.075500] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.076783] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.079767] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.097792] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.100783] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.104969] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 4.138137] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 4.141394] systemd[1]: Stopped target Basic System. worker2 # [ 4.144162] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 4.145875] systemd[1]: Stopped target Path Units. worker2 # [ 4.147876] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 4.152167] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 4.156629] systemd[1]: Stopped target Slice Units. worker2 # [ 4.157440] systemd[1]: Stopped target Socket Units. worker2 # [ 4.160778] systemd[1]: Stopped target System Initialization. worker2 # [ 4.164577] systemd[1]: Stopped target Swaps. worker2 # [ 4.167644] systemd[1]: Stopped target Timer Units. worker2 # [ 4.170009] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 4.175808] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 4.177740] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 4.179508] systemd[1]: Stopped Find NixOS closure. worker2 # [ 4.188235] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker2 # [ 4.189236] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.190224] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 4.191161] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 4.191898] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 4.208911] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 4.211814] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 4.219111] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 4.221635] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 4.226000] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 4.229544] systemd[1]: Stopped target Local File Systems. worker2 # [ 4.231390] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 4.233810] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 4.236109] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 4.238871] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 4.240187] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.244464] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.245295] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker2 # [ 4.246307] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker2 # [ 4.247666] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.248853] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.249672] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 4.250601] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 4.256179] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 4.258186] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 4.260143] systemd[1]: systemd-udevd.service: Consumed 1.117s CPU time over 2.262s wall clock time, 18.3M memory peak. worker2 # [ 4.264497] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 4.266214] systemd[1]: Closed udev Control Socket. worker2 # [ 4.267044] systemd[1]: Starting Cleanup udev Database... worker2 # [ 4.267833] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.269075] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 4.269943] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 4.271051] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 4.276122] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 4.277252] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 4.307932] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 4.312990] systemd[1]: Finished Cleanup udev Database. worker2 # [ 4.313819] systemd[1]: Reached target Switch Root. worker2 # [ 4.314527] systemd[1]: Starting NixOS Activation... client # [ 4.357769] (udev-worker)[106]: mtd0ro: Failed to find and pin callout binary "/nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.363934] (udev-worker)[106]: mtd0ro: /nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.401792] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.404203] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.367074] (udev-worker)[97]: mtd0ro: Failed to find and pin callout binary "/nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.405508] systemd[1]: Stopping Virtual Console Setup... client # [ 4.408238] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.374393] (udev-worker)[97]: mtd0ro: /nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.433200] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.397390] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.436815] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.400361] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.404144] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.405033] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.418045] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.419158] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.423926] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.441766] systemd-vconsole-setup[155]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.445052] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.475401] initrd-nixos-activation-start[172]: booting system configuration /nix/store/fhjd99g778w0n7n58v9pngxdn87svwlf-nixos-system-worker2-test lb # [ 4.479909] (udev-worker)[101]: mtd0ro: Failed to find and pin callout binary "/nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/mtd_probe": No such file or directory lb # [ 4.485755] (udev-worker)[101]: mtd0ro: /nix/store/awq6qdiyxlvdp21km39qj020fcshgkia-systemd-261.2/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.518591] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.520203] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.521737] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.524172] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.541693] initrd-nixos-activation-start[172]: running activation script... lb # [ 4.546419] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.548893] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.552179] systemd[1]: Starting Virtual Console Setup... lb # [ 4.577102] systemd-vconsole-setup[156]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.580480] systemd[1]: Finished Virtual Console Setup. client # [ 4.803456] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.803793] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.867029] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.872542] systemd[1]: Reached target Initrd File Systems. client # [ 4.878437] systemd[1]: Starting Find NixOS closure... client # [ 4.888450] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.865723] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.871305] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.874320] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.884413] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.931967] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.907529] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.940421] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.952855] systemd[1]: Finished Find NixOS closure. client # [ 4.955660] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.923786] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.960506] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.926486] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.937525] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.939632] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.944305] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.991354] systemd[1]: Stopped target Initrd Default Target. client # [ 4.996743] systemd[1]: Stopped target Basic System. client # [ 4.997792] systemd[1]: Stopped target Initrd Root Device. client # [ 4.998876] systemd[1]: Stopped target Path Units. worker1 # [ 4.963403] systemd[1]: Stopped target Initrd Default Target. client # [ 4.999847] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.965304] systemd[1]: Stopped target Basic System. worker1 # [ 4.966318] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 4.975119] initrd-nixos-activation-start[195]: setting up /etc... lb # [ 4.975642] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.968440] systemd[1]: Stopped target Path Units. client # [ 5.008296] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.972366] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.980485] systemd[1]: Reached target Initrd File Systems. client # [ 5.009723] systemd[1]: Stopped target Slice Units. client # [ 5.010611] systemd[1]: Stopped target Socket Units. worker1 # [ 4.974273] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.011505] systemd[1]: Stopped target System Initialization. client # [ 5.013522] systemd[1]: Stopped target Swaps. client # [ 5.015119] systemd[1]: Stopped target Timer Units. worker1 # [ 4.979747] systemd[1]: Stopped target Slice Units. lb # [ 4.987484] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.980682] systemd[1]: Stopped target Socket Units. client # [ 5.017096] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.981403] systemd[1]: Stopped target System Initialization. worker1 # [ 4.982202] systemd[1]: Stopped target Swaps. worker1 # [ 4.982844] systemd[1]: Stopped target Timer Units. worker1 # [ 4.983541] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.020553] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.985372] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.986235] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 4.987287] systemd[1]: Stopped Find NixOS closure. client # [ 5.025096] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.996504] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 5.028510] systemd[1]: Stopped Find NixOS closure. client # [ 5.031303] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 4.999614] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 5.001780] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.002777] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.039954] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.003708] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.044553] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.045690] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.046688] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.047828] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.016300] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 5.017594] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.018443] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.019551] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.058804] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.061055] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.062224] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.063278] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 5.030840] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.031995] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 5.036184] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.038051] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.076276] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.040108] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.077162] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 5.041627] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.078239] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.080515] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.044626] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.046471] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.054474] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 5.084246] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 5.085415] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.056834] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 5.087938] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.051870] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.052891] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. client # [ 5.089718] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.053909] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 5.054750] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.091697] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.055675] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.093194] systemd[1]: systemd-udevd.service: Consumed 1.402s CPU time over 3.016s wall clock time, 22.4M memory peak. worker1 # [ 5.057299] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.094621] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.058256] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.095549] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.059278] systemd[1]: systemd-udevd.service: Consumed 1.408s CPU time over 3.082s wall clock time, 25.5M memory peak. worker1 # [ 5.060870] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.061860] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.062675] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.063672] systemd[1]: Closed udev Control Socket. lb # [ 5.073598] systemd[1]: Finished Find NixOS closure. client # [ 5.104277] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 5.076236] systemd[1]: Reached target Initrd Default Target. client # [ 5.105406] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 5.106543] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.079019] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.072564] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.073437] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 5.074791] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.112197] systemd[1]: Closed udev Control Socket. worker1 # [ 5.075709] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.113010] systemd[1]: Starting Cleanup udev Database... client # [ 5.113803] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.114855] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.115718] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.080202] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.081203] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.082159] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.120510] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.121506] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.122461] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.123317] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 5.128198] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.114289] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.119117] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.111250] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.120617] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.112865] systemd[1]: Finished Cleanup udev Database. client # [ 5.147148] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.121763] systemd[1]: Stopped target Basic System. worker1 # [ 5.113673] systemd[1]: Reached target Switch Root. client # [ 5.150488] systemd[1]: Finished Cleanup udev Database. client # [ 5.151252] systemd[1]: Reached target Switch Root. worker1 # [ 5.116279] systemd[1]: Starting NixOS Activation... lb # [ 5.124547] systemd[1]: Stopped target Initrd Root Device. lb # [ 5.127337] systemd[1]: Stopped target Path Units. client # [ 5.156373] systemd[1]: Starting NixOS Activation... lb # [ 5.128449] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 5.131266] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 5.136276] systemd[1]: Stopped target Slice Units. lb # [ 5.137181] systemd[1]: Stopped target Socket Units. lb # [ 5.138006] systemd[1]: Stopped target System Initialization. lb # [ 5.138927] systemd[1]: Stopped target Swaps. lb # [ 5.139671] systemd[1]: Stopped target Timer Units. lb # [ 5.144288] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 5.145261] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 5.146988] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 5.151768] systemd[1]: Stopped Find NixOS closure. lb # [ 5.154417] systemd[1]: Starting Load Kernel Module 9pnet_virtio... lb # [ 5.156325] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 5.158312] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.170848] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.171716] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 5.173242] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.179504] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.183013] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.186367] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.188222] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.190638] systemd[1]: Stopped target Local File Systems. lb # [ 5.191707] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.196636] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.197644] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.198396] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.199383] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.202200] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.202968] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.203943] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.204926] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 5.205840] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.206809] systemd[1]: systemd-udevd.service: Consumed 1.423s CPU time over 3.112s wall clock time, 22.2M memory peak. lb # [ 5.212328] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. lb # [ 5.213447] systemd[1]: Finished Load Kernel Module 9pnet_virtio. lb # [ 5.216452] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.220223] systemd[1]: Closed udev Control Socket. lb # [ 5.221060] systemd[1]: Starting Cleanup udev Database... lb # [ 5.221820] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.224171] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.228132] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.229280] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.230436] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 5.232219] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.241150] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.243898] systemd[1]: Finished NixOS Activation. worker2 # [ 5.248314] systemd[1]: Starting Switch Root... lb # [ 5.258757] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.261993] systemd[1]: Finished Cleanup udev Database. lb # [ 5.262833] systemd[1]: Reached target Switch Root. worker2 # [ 5.267910] systemd[1]: Switching root. lb # [ 5.268072] systemd[1]: Starting NixOS Activation... worker1 # [ 5.282192] initrd-nixos-activation-start[181]: booting system configuration /nix/store/h490qvdfgh8rdlrafpm8wgj56x94xnlb-nixos-system-worker1-test client # [ 5.326807] initrd-nixos-activation-start[181]: booting system configuration /nix/store/5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test worker1 # [ 5.347489] initrd-nixos-activation-start[181]: running activation script... client # [ 5.394004] initrd-nixos-activation-start[181]: running activation script... lb # [ 5.437582] initrd-nixos-activation-start[181]: booting system configuration /nix/store/a9n40kyqx09p2mx9kz6ck1ddzmmfmfqv-nixos-system-lb-test worker2 # [ 5.465099] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). lb # [ 5.503178] initrd-nixos-activation-start[181]: running activation script... worker1 # [ 5.766664] initrd-nixos-activation-start[204]: setting up /etc... client # [ 5.826206] initrd-nixos-activation-start[204]: setting up /etc... lb # [ 5.958890] initrd-nixos-activation-start[204]: setting up /etc... worker1 # [ 6.047744] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 6.050982] systemd[1]: Finished NixOS Activation. worker1 # [ 6.052316] systemd[1]: Starting Switch Root... client # [ 6.097562] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 6.100199] systemd[1]: Finished NixOS Activation. client # [ 6.104245] systemd[1]: Starting Switch Root... worker2 # [ 6.080735] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.093310] systemd[1]: Detected virtualization qemu. worker2 # [ 6.097335] systemd[1]: Detected architecture arm64. worker1 # [ 6.076993] systemd[1]: Switching root. worker2 # [ 6.099425] systemd[1]: Detected first boot. worker2 # [ 6.106473] systemd[1]: Initializing machine ID from random generator. client # [ 6.125329] systemd[1]: Switching root. lb # [ 6.215134] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 6.218019] systemd[1]: Finished NixOS Activation. lb # [ 6.220127] systemd[1]: Starting Switch Root... lb # [ 6.243915] systemd[1]: Switching root. worker1 # [ 6.263466] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). client # [ 6.309502] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). lb # [ 6.432814] systemd-journald[73]: Received SIGTERM from PID 1 (systemd). worker2 # [ 6.440600] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.632756] systemd[1]: Applying preset policy. client # [ 6.882667] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.890362] systemd[1]: Detected virtualization qemu. client # [ 6.894198] systemd[1]: Detected architecture arm64. client # [ 6.896383] systemd[1]: Detected first boot. client # [ 6.903375] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.865998] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.871207] systemd[1]: Detected virtualization qemu. worker1 # [ 6.873907] systemd[1]: Detected architecture arm64. worker1 # [ 6.876876] systemd[1]: Detected first boot. worker1 # [ 6.884973] systemd[1]: Initializing machine ID from random generator. lb # [ 7.037154] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 7.044939] systemd[1]: Detected virtualization qemu. lb # [ 7.047141] systemd[1]: Detected architecture arm64. lb # [ 7.049978] systemd[1]: Detected first boot. lb # [ 7.054999] systemd[1]: Initializing machine ID from random generator. worker2 # [ 7.128108] systemd[1]: Populated /etc with preset unit settings. client # [ 7.233507] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 7.209949] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 7.355687] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 7.428906] systemd[1]: Applying preset policy. worker1 # [ 7.403344] systemd[1]: Applying preset policy. lb # [ 7.544759] systemd[1]: Applying preset policy. worker2 # [ 7.633127] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.634795] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.637895] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.641598] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.643657] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.646009] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.647459] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.649892] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.651632] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.653529] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.655045] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.656446] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.658016] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.660211] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.661443] systemd[1]: Reached target Path Units. worker2 # [ 7.663516] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.664545] systemd[1]: Reached target Slice Units. worker2 # [ 7.666014] systemd[1]: Reached target Swaps. worker2 # [ 7.670381] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.675147] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.679127] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.682984] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.684190] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.690156] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.692344] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.696519] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.698137] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.699952] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.701966] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.713619] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.714092] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.714447] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.720860] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.725269] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.730488] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.745793] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.758924] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.778688] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker2 # [ 7.780752] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 7.796994] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.799720] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.802943] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.813259] systemd[1]: Starting Load Kernel Module fuse... worker2 # [ 7.818405] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 7.869590] systemd[1]: Starting Journal Service... client # [ 7.909364] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.881885] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.888238] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.898201] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.913461] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.917433] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.929958] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.938873] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.941219] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.941699] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.942097] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.942469] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.942852] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.981413] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 7.990203] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 8.038314] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 8.065457] EXT4-fs (vda): re-mounted 47eff782-01e5-4cab-8c65-8fec002074a1. worker2 # [ 8.074172] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 8.074762] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 8.075054] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 8.083875] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 8.086334] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.093609] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker2 # [ 8.096168] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker2 # [ 8.117289] systemd[1]: Finished Load Kernel Modules. worker2 # [ 8.122452] systemd[1]: Starting Firewall... worker2 # [ 8.127293] systemd-journald[267]: Collecting audit messages is enabled. worker2 # [ 8.148338] fuse: init (API version 7.45) worker2 # [ 8.158033] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 8.168065] systemd[1]: Started Journal Service. worker2 # [ 8.157599] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 8.165410] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 8.173219] systemd-modules-load[268]: Module 'atkbd' is built in worker2 # [ 8.186417] systemd-modules-load[268]: Module 'loop' is built in worker2 # [ 8.203470] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 8.216423] systemd[1]: modprobe@fuse.service: Deactivated successfully. worker2 # [ 8.228219] systemd[1]: Finished Load Kernel Module fuse. worker2 # [ 8.241176] systemd-oomd[269]: No swap; memory pressure usage will be degraded worker2 # [ 8.252877] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.259454] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 8.268257] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.289593] systemd-journald[267]: Received client request to flush runtime journal. worker1 # [ 8.344787] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 8.346101] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 8.349582] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 8.354125] systemd[1]: Created slice Slice /system/getty. worker1 # [ 8.357076] systemd[1]: Created slice User and Session Slice. worker2 # [ 8.346982] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 8.358377] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 8.353725] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.360370] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 8.397226] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 8.362881] systemd[1]: Expecting device /dev/hvc0... client # [ 8.398611] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 8.363947] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 8.359014] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 8.365445] systemd[1]: Reached target Local Encrypted Volumes. client # [ 8.402214] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 8.367886] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 8.369781] systemd[1]: Stopped target initrd-root-fs.target. client # [ 8.407186] systemd[1]: Created slice Slice /system/getty. worker1 # [ 8.371719] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 8.365780] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.373737] systemd[1]: Reached target Virtual Machines and Containers. client # [ 8.409773] systemd[1]: Created slice User and Session Slice. worker1 # [ 8.375860] systemd[1]: Reached target Path Units. client # [ 8.411782] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 8.377445] systemd[1]: Reached target Remote File Systems. client # [ 8.412952] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 8.415544] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 8.379344] systemd[1]: Reached target Slice Units. client # [ 8.416291] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 8.381312] systemd[1]: Reached target Swaps. client # [ 8.417429] systemd[1]: Reached target Local Encrypted Volumes. client # [ 8.418637] systemd[1]: Stopped target initrd-fs.target. client # [ 8.420150] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 8.385422] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 8.421451] systemd[1]: Stopped target initrd-switch-root.target. client # [ 8.423534] systemd[1]: Reached target Virtual Machines and Containers. client # [ 8.425556] systemd[1]: Reached target Path Units. worker1 # [ 8.390443] systemd[1]: Listening on Process Core Dump Socket. client # [ 8.427539] systemd[1]: Reached target Remote File Systems. client # [ 8.429501] systemd[1]: Reached target Slice Units. worker1 # [ 8.394586] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 8.431457] systemd[1]: Reached target Swaps. worker1 # [ 8.398787] systemd[1]: Listening on Factory Reset Management. client # [ 8.435970] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 8.401013] systemd[1]: Listening on Hostname Service Socket. client # [ 8.441491] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 8.406792] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 8.410110] systemd[1]: Listening on Journal Audit Socket. client # [ 8.445990] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 8.414801] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 8.450467] systemd[1]: Listening on Factory Reset Management. worker1 # [ 8.416504] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 8.452789] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 8.419397] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.421150] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 8.459412] systemd[1]: Starting Journal Log Access Socket... client # [ 8.462149] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 8.430635] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 8.431089] systemd[1]: Listening on udev Control Socket. client # [ 8.466889] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 8.431434] systemd[1]: Listening on udev Varlink Socket. client # [ 8.469911] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 8.436792] systemd[1]: Mounting Huge Pages File System... client # [ 8.472912] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.474610] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 8.444124] systemd[1]: Mounting POSIX Message Queue File System... client # [ 8.486705] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 8.448848] systemd[1]: Mounting Kernel Debug File System... client # [ 8.488274] systemd[1]: Listening on udev Control Socket. client # [ 8.490891] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 8.452587] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.453844] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.496232] systemd[1]: Mounting Huge Pages File System... worker1 # [ 8.463086] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 8.457736] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.502214] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 8.477030] systemd[1]: Starting Create List of Static Device Nodes... client # [ 8.514076] systemd[1]: Mounting Kernel Debug File System... client # [ 8.520784] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 8.490936] systemd[1]: Starting Load Kernel Module 9pnet_virtio... worker1 # [ 8.495996] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 8.540174] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 8.510910] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 8.511315] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 8.511593] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 8.554520] systemd[1]: Starting Load Kernel Module 9pnet_virtio... client # [ 8.557475] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 8.539304] systemd[1]: Starting Load Kernel Module fuse... lb # [ 8.549964] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 8.551303] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 8.543681] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 8.581703] systemd[1]: Mounting Kernel Configuration File System... lb # [ 8.554600] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 8.558092] systemd[1]: Created slice Slice /system/getty. lb # [ 8.559668] systemd[1]: Created slice User and Session Slice. lb # [ 8.560603] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 8.589696] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 8.562615] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 8.563988] systemd[1]: Expecting device /dev/hvc0... lb # [ 8.565200] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 8.566659] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 8.567982] systemd[1]: Stopped target initrd-fs.target. lb # [ 8.569216] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 8.570630] systemd[1]: Stopped target initrd-switch-root.target. client # [ 8.599110] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 8.572451] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 8.574363] systemd[1]: Reached target Path Units. lb # [ 8.575915] systemd[1]: Reached target Remote File Systems. lb # [ 8.577552] systemd[1]: Reached target Slice Units. lb # [ 8.579195] systemd[1]: Reached target Swaps. lb # [ 8.583554] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 8.588653] systemd[1]: Listening on Process Core Dump Socket. lb # [ 8.593022] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 8.596643] systemd[1]: Listening on Factory Reset Management. lb # [ 8.597905] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 8.592391] systemd[1]: Starting Journal Service... lb # [ 8.603280] systemd[1]: Starting Journal Log Access Socket... lb # [ 8.605559] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 8.603143] systemd[1]: Starting Load Kernel Modules... lb # [ 8.609962] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 8.612622] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 8.644012] systemd[1]: Starting Load Kernel Module fuse... lb # [ 8.615359] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 8.618016] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 8.652654] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 8.622648] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 8.629846] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 8.630360] systemd[1]: Listening on udev Control Socket. lb # [ 8.630736] systemd[1]: Listening on udev Varlink Socket. lb # [ 8.637209] systemd[1]: Mounting Huge Pages File System... worker2 # [ 8.624652] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.632333] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 8.628517] systemd[1]: Mounting FUSE Control File System... lb # [ 8.641620] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 8.634911] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.681424] systemd[1]: Starting Journal Service... lb # [ 8.652605] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 8.649978] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 8.654774] systemd[1]: Listening on Journal Log Access Socket. client # [ 8.691796] systemd[1]: Starting Load Kernel Modules... lb # [ 8.664679] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 8.660299] systemd[1]: Mounted Huge Pages File System. worker1 # [ 8.664325] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 8.664810] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 8.665204] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 8.665586] systemd[1]: Mounted Kernel Configuration File System. lb # [ 8.680831] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 8.672971] systemd[1]: Mounted FUSE Control File System. lb # [ 8.690725] systemd[1]: Starting Load Kernel Module 9pnet_virtio... lb # [ 8.692923] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 8.724519] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 8.708907] systemd[1]: Mounting Kernel Configuration File System... lb # [ 8.710523] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 8.718108] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 8.713197] systemd[1]: Finished Create List of Static Device Nodes. client # [ 8.751504] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 8.715969] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.722353] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 8.717416] systemd[1]: Reached target Local File Systems. lb # [ 8.738417] systemd[1]: Starting Load Kernel Module fuse... worker2 # [ 8.723669] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.771786] fuse: init (API version 7.45) lb # [ 8.741973] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 8.731172] systemd[1]: Starting register-nix-paths.service... client # [ 8.775299] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.734397] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 8.740982] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.748973] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.761572] systemd-udevd[305]: Using default interface naming scheme 'v261'. client # [ 8.803503] systemd-journald[275]: Collecting audit messages is enabled. worker2 # [ 8.768784] systemd[1]: Starting Create System Files and Directories... client # [ 8.812588] systemd[1]: Starting Coldplug All udev Devices... client # [ 8.817479] systemd[1]: Started Journal Service. client # [ 8.808043] systemd[1]: Queued start job for default target Multi-User System. client # [ 8.809368] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 8.810740] systemd-modules-load[276]: Module 'atkbd' is built in client # [ 8.811733] systemd-modules-load[276]: Module 'loop' is built in client # [ 8.824521] systemd[1]: Listening on Journal Log Access Socket. client # [ 8.825561] systemd[1]: Mounted Huge Pages File System. worker1 # [ 8.803997] EXT4-fs (vda): re-mounted bf9ef386-e4ac-4152-8954-4575120e86c0. client # [ 8.826477] systemd[1]: Mounted POSIX Message Queue File System. client # [ 8.827456] systemd[1]: Mounted Kernel Debug File System. lb # [ 8.815453] systemd[1]: Starting Journal Service... worker1 # [ 8.813992] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 8.838086] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 8.816764] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 8.841248] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 8.819007] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 8.844818] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 8.825470] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 8.834042] systemd[1]: Starting Load Kernel Modules... worker1 # [ 8.828456] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.850472] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 8.832123] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. client # [ 8.854931] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 8.834767] systemd[1]: Finished Load Kernel Module 9pnet_virtio. client # [ 8.861294] systemd[1]: modprobe@fuse.service: Deactivated successfully. client # [ 8.868701] systemd[1]: Finished Load Kernel Module fuse. client # [ 8.870970] systemd[1]: Finished Load Kernel Modules. worker1 # [ 8.856622] systemd[1]: Finished Load Kernel Modules. worker2 # [ 8.856541] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.866278] systemd-journald[275]: Collecting audit messages is enabled. worker2 # [ 8.862494] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.897855] systemd[1]: Mounting FUSE Control File System... lb # [ 8.882797] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 8.900201] systemd-oomd[277]: No swap; memory pressure usage will be degraded worker1 # [ 8.885950] systemd[1]: Starting Firewall... client # [ 8.907680] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 8.897129] systemd[1]: Starting Apply Kernel Variables... client # [ 8.943936] EXT4-fs (vda): re-mounted d4dd8969-73c0-43cf-ab04-047d2883e63d. client # [ 8.932281] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 8.917882] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 8.910287] systemd[1]: Started Journal Service. client # [ 8.936037] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.916265] fuse: init (API version 7.45) lb # [ 8.925532] systemd-journald[275]: Collecting audit messages is enabled. worker1 # [ 8.910076] systemd[1]: Queued start job for default target Multi-User System. lb # [ 8.933068] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.954560] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 8.927341] systemd[1]: Queued start job for default target Multi-User System. client # [ 8.957460] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.911672] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 8.951126] fuse: init (API version 7.45) worker1 # [ 8.925790] systemd-modules-load[276]: Module 'atkbd' is built in client # [ 8.970537] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 8.959280] systemd[1]: Starting Coldplug All udev Devices... client # [ 8.976368] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 8.942420] systemd-modules-load[276]: Module 'loop' is built in client # [ 8.987655] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 8.953652] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 8.977865] systemd[1]: Started Journal Service. client # [ 9.006554] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.966969] systemd[1]: modprobe@fuse.service: Deactivated successfully. worker1 # [ 8.978393] systemd[1]: Finished Load Kernel Module fuse. lb # [ 9.007787] EXT4-fs (vda): re-mounted 263eff33-f374-436f-b58d-fd57b4da33c1. worker1 # [ 8.986385] systemd-oomd[277]: No swap; memory pressure usage will be degraded worker1 # [ 8.995706] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.999770] systemd[1]: Finished Create System Files and Directories. worker2 # [ 9.012285] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 9.007872] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 9.017357] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 9.018927] systemd[1]: Reached target First Boot Complete. worker1 # [ 9.059672] systemd-journald[275]: Received client request to flush runtime journal. client # [ 9.088248] systemd[1]: Mounted FUSE Control File System. client # [ 9.111698] systemd-journald[275]: Received client request to flush runtime journal. worker2 # [ 9.073492] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 9.117420] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 9.122427] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 9.126258] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 9.129307] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 9.168687] systemd[1]: Finished Apply Kernel Variables. client # [ 9.169627] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 9.170632] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 9.143060] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.171563] systemd[1]: Finished Load/Save OS Random Seed. client # [ 9.177924] systemd[1]: Reached target First Boot Complete. client # [ 9.181855] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 9.190934] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 9.192742] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 9.197126] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 9.254693] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 9.255812] systemd[1]: Reached target Preparation for Local File Systems. client # [ 9.258500] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 9.257087] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 9.264390] systemd[1]: Starting Update is Completed... worker1 # [ 9.317562] systemd-udevd[311]: Using default interface naming scheme 'v261'. client # [ 9.364202] systemd-udevd[307]: Using default interface naming scheme 'v261'. worker1 # [ 9.334251] systemd[1]: Mounting /run/wrappers... worker1 # [ 9.344601] systemd[1]: Mounting FUSE Control File System... client # [ 9.389075] systemd[1]: Mounting /run/wrappers... worker1 # [ 9.376798] systemd[1]: Mounted FUSE Control File System. worker2 # [ 9.395107] systemd[1]: Finished Update is Completed. client # [ 9.431560] systemd[1]: Mounted /run/wrappers. client # [ 9.433079] systemd[1]: Reached target Local File Systems. client # [ 9.438483] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 9.403659] systemd[1]: Mounted /run/wrappers. worker1 # [ 9.405263] systemd[1]: Reached target Local File Systems. client # [ 9.442503] systemd[1]: Starting register-nix-paths.service... worker1 # [ 9.408328] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 9.446094] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 9.414872] systemd[1]: Starting register-nix-paths.service... client # [ 9.447650] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 9.456567] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 9.422544] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 9.424432] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 9.437056] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 9.477734] systemd[1]: Starting Create System Files and Directories... worker1 # [ 9.449011] systemd[1]: Starting Create System Files and Directories... worker1 # [ 9.530892] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 9.539760] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 9.576608] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 8.930157] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 9.582839] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 9.570154] systemd-journald[275]: Received client request to flush runtime journal. client # [ 9.645336] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 9.639130] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 9.653506] systemd-modules-load[276]: Module 'atkbd' is built in client # [ 9.681806] systemd[1]: Finished Create System Files and Directories. lb # [ 9.661663] systemd-modules-load[276]: Module 'loop' is built in lb # [ 9.664465] systemd[1]: Listening on Journal Log Access Socket. lb # [ 9.669508] systemd[1]: Mounted Huge Pages File System. client # [ 9.699212] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 9.674856] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 9.676781] systemd[1]: Mounted Kernel Debug File System. client # [ 9.710055] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 9.683596] systemd[1]: Mounted Kernel Trace File System. lb # [ 9.684897] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 9.690394] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. worker1 # [ 9.686237] systemd[1]: Finished Create System Files and Directories. lb # [ 9.699648] systemd[1]: Finished Load Kernel Module 9pnet_virtio. worker1 # [ 9.693048] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 9.704509] systemd[1]: Mounted Kernel Configuration File System. lb # [ 9.705519] systemd[1]: modprobe@fuse.service: Deactivated successfully. lb # [ 9.706589] systemd[1]: Finished Load Kernel Module fuse. lb # [ 9.707480] systemd[1]: Finished Load Kernel Modules. worker1 # [ 9.702591] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 9.718418] systemd[1]: Mounting FUSE Control File System... lb # [ 9.735137] systemd[1]: Starting Firewall... lb # [ 9.742870] systemd[1]: Starting Apply Kernel Variables... lb # [ 9.750714] systemd-oomd[278]: No swap; memory pressure usage will be degraded lb # [ 9.759962] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 9.774380] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 9.814051] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 9.785189] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 9.794514] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 9.808765] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 9.817300] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 9.818949] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 9.819763] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 9.839714] systemd[1]: Finished Apply Kernel Variables. client # [ 9.870426] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 9.834769] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 9.847803] systemd[1]: Mounted FUSE Control File System. lb # [ 9.849588] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 9.853484] systemd[1]: Reached target First Boot Complete. client # [ 9.886184] systemd[1]: Starting Update is Completed... lb # [ 9.868717] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 9.869835] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 9.870914] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 9.871812] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 9.888826] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 9.904207] systemd-udevd[304]: Using default interface naming scheme 'v261'. worker2 # [ 9.905970] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.909875] systemd[1]: Mounting /run/wrappers... worker2 # [ 9.910855] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.920615] systemd[1]: Mounted /run/wrappers. lb # [ 9.921329] systemd[1]: Reached target Local File Systems. lb # [ 9.922128] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 9.923061] systemd[1]: Starting register-nix-paths.service... lb # [ 9.923891] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 9.923903] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 9.926856] systemd[1]: Starting Update is Completed... client # [ 9.963647] systemd[1]: Finished Update is Completed. lb # [ 9.936313] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 9.946264] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 9.954800] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 9.956888] systemd[1]: Starting Create System Files and Directories... lb # [ 9.963173] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 9.975120] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 10.009773] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 10.005038] systemd[1]: Finished Update is Completed. lb # [ 10.067207] systemd[1]: Finished Create System Files and Directories. lb # [ 10.083214] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 10.108217] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 lb # [ 10.119299] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 10.242867] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 10.286033] systemd[1]: Finished Rebuild Journal Catalog. client # [ 10.315100] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 10.318815] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 10.298315] systemd[1]: Starting Update is Completed... worker2 # [ 10.342480] systemd[1]: Finished Firewall. lb # [ 10.378315] systemd[1]: Finished Update is Completed. worker1 # [ 10.463412] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 10.467592] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 10.612986] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.673108] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.714869] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 10.789802] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 10.796356] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 10.921934] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.924903] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.925929] systemd[1]: Reached target System Initialization. worker2 # [ 10.928205] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 10.930620] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.931585] systemd[1]: Reached target Timer Units. worker2 # [ 10.936842] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.939673] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.940798] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 10.941955] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.947856] systemd[1]: Reached target Socket Units. client # [ 10.980285] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.954305] systemd[1]: Reached target Basic System. worker2 # [ 10.960983] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.953240] systemd[1]: Finished Firewall. worker2 # [ 10.968738] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.978102] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.985496] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.993119] systemd[1]: Starting resolvconf update... worker2 # [ 11.001945] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 11.082899] systemd[1]: Finished Post-Boot Actions. client # [ 11.158518] systemd[1]: Finished register-nix-paths.service. client # [ 11.160212] systemd[1]: Reached target System Initialization. worker2 # [ 11.132303] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.164272] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 11.166161] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 11.168907] systemd[1]: Reached target Timer Units. worker2 # [ 11.141361] nsncd[490]: Sep 22 10:49:20.735 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.172532] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 11.173522] systemd[1]: Listening on Nix Daemon Socket. client # [ 11.179072] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 11.180463] systemd[1]: Reached target Socket Units. worker2 # [ 11.154409] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.146165] systemd[1]: Finished Coldplug All udev Devices. client # [ 11.183224] systemd[1]: Reached target Basic System. worker2 # [ 11.155863] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.189310] systemd[1]: Started backdoor.service. client # [ 11.190964] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 11.166102] systemd[1]: Starting User Login Management... client # [ 11.202690] systemd[1]: Starting nftables firewall... worker2 # [ 11.171950] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 11.221772] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 11.242986] systemd[1]: Starting Post-Boot Actions... lb # [ 11.217369] systemd[1]: Finished register-nix-paths.service. worker2 # [ 11.224441] dbus-broker-launch[494]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 11.217992] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 11.269150] systemd[1]: Started Reset console on configuration changes. worker2 # [ 11.247929] dbus-broker-launch[494]: NSS returned no entry for 'systemd-timesync' lb # [ 11.258687] systemd[1]: Finished Coldplug All udev Devices. lb # [ 11.259711] systemd[1]: Reached target System Initialization. lb # [ 11.260785] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 11.251422] dbus-broker-launch[494]: Invalid user-name in /nix/store/a9ijqx61hpssavy5d9lh27li51m8hnl7-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 11.264423] systemd[1]: Started niks3 garbage collection timer. lb # [ 11.269472] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 11.270483] systemd[1]: Reached target Timer Units. lb # [ 11.272277] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 11.301079] systemd[1]: Starting resolvconf update... lb # [ 11.279005] systemd[1]: Listening on niks3 server socket. lb # [ 11.280204] systemd[1]: Listening on Nix Daemon Socket. lb # [ 11.281329] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 11.292151] systemd[1]: Reached target Socket Units. worker2 # [ 11.294364] systemd[1]: Started D-Bus System Message Bus. lb # [ 11.296222] systemd[1]: Reached target Basic System. lb # [ 11.297687] systemd[1]: Starting Import lastlog data into lastlog2 database... client # connecting to host... lb # [ 11.300948] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.311017] systemd[1]: Starting Post-Boot Actions... worker2 # [ 11.330773] systemd-logind[503]: New seat seat0. lb # [ 11.336244] systemd[1]: Started Reset console on configuration changes. worker2 # [ 11.342580] systemd[1]: Started User Login Management. client: Guest shell says: b'Spawning backdoor root shell...\n' worker2 # [ 11.347023] systemd[1]: Starting linger-users.service... lb # [ 11.356881] systemd[1]: Starting resolvconf update... client # [ 11.379940] nsncd[398]: Sep 22 10:49:20.977 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 11.361426] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.391617] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 11.368607] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 11.406983] systemd[1]: Started rustfs.service. worker2 # [ 11.376334] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 11.412660] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 11.446729] systemd[1]: Starting rustfs-bucket.service... client: connected to guest root shell worker1 # [ 11.420140] systemd[1]: Finished register-nix-paths.service. client # [ 11.413580] systemd[1]: Finished Post-Boot Actions. worker2 # [ 11.382042] systemd[1]: Stopping User and Group Name Lookups... client: (connecting took 11.89 seconds) worker1 # [ 11.423815] systemd[1]: Reached target System Initialization. worker2 # [ 11.383916] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client: (finished: waiting for the VM to finish booting, in 11.89 seconds) worker1 # [ 11.426154] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 11.435460] (udev-worker)[328]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 11.429050] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 11.446667] (udev-worker)[328]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.434809] systemd[1]: Reached target Timer Units. client # [ 11.454678] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.437226] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 11.462095] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.443869] systemd[1]: Listening on Nix Daemon Socket. client # [ 11.463435] (udev-worker)[384]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.452648] systemd[1]: Listening on Nix worker-protocol over gRPC. client # [ 11.471779] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 11.455655] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 11.490053] systemd[1]: Starting User Login Management... client # [ 11.495528] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 11.390890] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 11.393373] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 11.463194] systemd[1]: Reached target Socket Units. worker2 # [ 11.398338] dbus-broker-launch[494]: Ready worker2 # [ 11.403172] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 11.456113] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 11.460356] systemd[1]: Finished linger-users.service. worker2 # [ 11.473212] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 11.476554] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.470902] systemd[1]: Reached target Basic System. worker2 # [ 11.481060] nsncd[558]: Sep 22 10:49:21.077 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 11.485636] systemd[1]: Finished Post-Boot Actions. worker2 # [ 11.487462] systemd[1]: Reached target User and Group Name Lookups. lb # [ 11.492406] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.475199] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 11.486493] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.500146] nsncd[494]: Sep 22 10:49:21.065 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 11.498059] systemd[1]: Starting Post-Boot Actions... worker2 # [ 11.517301] systemd[1]: Finished resolvconf update. worker2 # [ 11.518341] systemd[1]: Reached target Preparation for Network. worker1 # [ 11.512173] systemd[1]: Started Reset console on configuration changes. worker2 # [ 11.523093] systemd[1]: Starting DHCP Client... lb # [ 11.525840] systemd[1]: Finished Firewall. worker1 # [ 11.519199] systemd[1]: Starting resolvconf update... worker2 # [ 11.528076] systemd[1]: Starting Extra networking commands.... worker1 # [ 11.529177] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 11.544610] systemd[1]: Started backdoor.service. lb # [ 11.557479] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.554548] systemd[1]: Starting D-Bus System Message Bus... lb # [ 11.563232] systemd[1]: Reached target User and Group Name Lookups. lb # [ 11.576166] systemd[1]: Starting D-Bus System Message Bus... lb # [ 11.584268] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 11.588120] systemd[1]: Starting Virtual Console Setup... worker2 # [ 11.612513] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 11.618350] systemd[1]: Stopped Virtual Console Setup. lb # [ 11.620618] systemd[1]: Starting User Login Management... worker1 # [ 11.614778] systemd[1]: Finished Post-Boot Actions. worker2 # [ 11.624994] systemd[1]: Starting Virtual Console Setup... lb # [ 11.629649] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 11.657290] nsncd[496]: Sep 22 10:49:21.259 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 11.675479] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.716750] dbus-broker-launch[426]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 11.695065] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 11.690953] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.692976] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.696538] systemd[1]: Starting User Login Management... worker1 # connecting to host... client # [ 11.739373] dbus-broker-launch[426]: NSS returned no entry for 'systemd-timesync' client # [ 11.748130] dbus-broker-launch[426]: Invalid user-name in /nix/store/ydh4ywywm30g15n5m6x5vfz42682s2cc-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 11.718125] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 11.793142] systemd-logind[430]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.804350] systemd-logind[430]: New seat seat0. client # [ 11.805124] systemd[1]: Started D-Bus System Message Bus. client # [ 11.805877] systemd[1]: Started User Login Management. worker1 # [ 11.777328] dbus-broker-launch[502]: Looking up NSS user entry for 'systemd-timesync'... client # [ 11.817577] systemd[1]: Starting linger-users.service... worker1 # [ 11.797366] dbus-broker-launch[502]: NSS returned no entry for 'systemd-timesync' worker1 # [ 11.802007] dbus-broker-launch[502]: Invalid user-name in /nix/store/a9ijqx61hpssavy5d9lh27li51m8hnl7-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 11.822098] dhcpcd[597]: dhcpcd-10.3.2 starting worker2 # [ 11.836379] dhcpcd[636]: dev: loaded udev client # [ 11.868114] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.869170] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.869995] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 11.842331] dhcpcd[636]: no valid interfaces found lb # [ 11.842390] dbus-broker-launch[528]: Looking up NSS user entry for 'systemd-timesync'... client # [ 11.870825] systemd[1]: Stopping User and Group Name Lookups... client # [ 11.871625] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 11.844758] dhcpcd[636]: no valid interfaces found worker1 # [ 11.846470] systemd[1]: Started D-Bus System Message Bus. client # [ 11.891725] dbus-broker-launch[426]: Ready lb # [ 11.863775] dbus-broker-launch[528]: NSS returned no entry for 'systemd-timesync' client # [ 11.897202] systemd[1]: nscd.service: Deactivated successfully. client # [ 11.900483] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 11.871532] dbus-broker-launch[528]: Invalid user-name in /nix/store/kxmx3smba321l9v2dpmh9zahhsbc77yh-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 11.881800] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.886680] systemd[1]: Reached target Network. worker2 # [ 11.890027] systemd[1]: Starting Permit User Sessions... client # [ 11.915183] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.921049] systemd[1]: Finished linger-users.service. worker1 # [ 11.892892] systemd-logind[520]: New seat seat0. lb # [ 11.901076] systemd-logind[531]: New seat seat0. worker1 # [ 11.902712] dbus-broker-launch[502]: Ready lb # [ 11.911693] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 11.903405] systemd[1]: Started User Login Management. worker1 # [ 11.911141] systemd[1]: Starting linger-users.service... client # [ 11.947579] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.921840] systemd[1]: Started User Login Management. lb # [ 11.923707] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 11.930334] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 11.927740] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 11.937983] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 11.931416] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 11.943486] systemd[1]: Finished Permit User Sessions. lb # [ 11.945251] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 11.938538] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 11.949398] systemd[1]: Started Getty on tty1. worker2 # [ 11.951145] systemd[1]: Reached target Login Prompts. lb # [ 11.953664] systemd[1]: Starting linger-users.service... worker1 # [ 11.945516] systemd[1]: Stopping User and Group Name Lookups... lb # [ 11.955730] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 11.950095] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 11.961436] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 11.957524] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 11.958408] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 11.959359] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.966670] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 11.991297] dbus-broker-launch[528]: Ready client # [ 12.026924] nsncd[504]: Sep 22 10:49:21.623 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 12.037288] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 12.038174] systemd[1]: Finished resolvconf update. client # [ 12.038851] systemd[1]: Reached target Host and Network Name Lookups. client # [ 12.039701] systemd[1]: Reached target User and Group Name Lookups. client # [ 12.048845] systemd[1]: Starting DHCP Client... worker1 # [ 12.024812] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 12.025814] systemd[1]: Finished linger-users.service. lb # [ 12.043390] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 12.071661] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 12.037717] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 12.041123] nsncd[569]: Sep 22 10:49:21.638 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 12.043207] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 12.048561] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 12.076281] systemd[1]: Finished resolvconf update. worker1 # [ 12.078555] systemd[1]: Reached target Preparation for Network. worker1 # [ 12.082903] systemd[1]: Starting DHCP Client... lb # [ 12.094573] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 12.088228] systemd[1]: Starting Extra networking commands.... lb # [ 12.101632] systemd[1]: Finished linger-users.service. worker2 # [ 12.101877] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 12.170052] mousedev: PS/2 mouse device common for all mice lb # [ 12.154717] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 12.161873] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 12.162181] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 12.167244] systemd[1]: Started backdoor.service. lb # [ 12.169421] nsncd[585]: Sep 22 10:49:21.736 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 12.168361] (udev-worker)[363]: Network interface NamePolicy= disabled on kernel command line. lb # [ 12.176801] systemd[1]: Reached target User and Group Name Lookups. lb # [ 12.180274] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 12.186413] (udev-worker)[355]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 12.202485] systemd[1]: Finished resolvconf update. lb # [ 12.205484] systemd[1]: Reached target Preparation for Network. worker1 # [ 12.201897] (udev-worker)[355]: Network interface NamePolicy= disabled on kernel command line. lb # [ 12.216508] systemd[1]: Starting DHCP Client... worker2 # [ 12.244805] 8021q: 802.1Q VLAN Support v1.8 lb # [ 12.233231] systemd[1]: Starting Extra networking commands.... client # [ 12.265456] dhcpcd[533]: dhcpcd-10.3.2 starting client # [ 12.280617] dhcpcd[543]: dev: loaded udev worker2 # [ 12.258203] systemd-vconsole-setup[595]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 12.273070] systemd[1]: Finished Virtual Console Setup. client # [ 12.315607] systemd[1]: Finished nftables firewall. lb # [ 12.290395] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 12.319469] systemd[1]: Reached target Preparation for Network. client # [ 12.325936] systemd-logind[430]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 12.330098] systemd[1]: Starting Address configuration of eth1... lb # [ 12.304771] systemd[1]: Started backdoor.service. client # [ 12.348181] 8021q: 802.1Q VLAN Support v1.8 client # [ 12.349028] systemd[1]: Starting Extra networking commands.... worker2 # [ 12.335413] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # connecting to host... worker2 # [ 12.341378] (udev-worker)[657]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 12.346431] dhcpcd[603]: dhcpcd-10.3.2 starting worker2 # [ 12.355678] (udev-worker)[655]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 12.372839] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 12.374945] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 12.378544] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 12.378870] cfg80211: failed to load regulatory.db worker2 # [ 12.368847] (udev-worker)[657]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 12.362272] dhcpcd[644]: dev: loaded udev worker1 # [ 12.382069] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 12.389907] systemd[1]: Starting Address configuration of eth1... worker1 # [ 12.420976] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 12.427532] systemd[1]: Finished Extra networking commands.. client # [ 12.484093] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 12.435435] systemd[1]: Reached target Network. worker1 # [ 12.441248] systemd[1]: Starting Permit User Sessions... worker2 # [ 12.455077] systemd-logind[503]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 12.500953] 8021q: adding VLAN 0 to HW filter on device eth1 lb # connecting to host... client # [ 12.511736] network-addresses-eth1-start[547]: adding address 192.168.1.1/24... done client # [ 12.532815] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 12.533302] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 12.539071] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 12.539416] cfg80211: failed to load regulatory.db worker1 # [ 12.508348] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 12.540445] network-addresses-eth1-start[547]: adding address 2001:db8:1::1/64... done worker2 # [ 12.520388] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 12.524748] systemd[1]: Starting Address configuration of eth1... worker1 # [ 12.525883] systemd[1]: Finished Permit User Sessions. worker1 # [ 12.543661] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 12.544230] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 12.549696] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 12.550022] cfg80211: failed to load regulatory.db worker1 # [ 12.540518] systemd[1]: Started Getty on tty1. worker1 # [ 12.541261] systemd[1]: Reached target Login Prompts. client # [ 12.578156] systemd[1]: Finished Address configuration of eth1. lb # [ 12.584989] dhcpcd[624]: dhcpcd-10.3.2 starting client # [ 12.615205] dhcpcd[594]: /nix/store/grcr27glg0bai58hwyh3zxlj5sf30hhx-openresolv-3.17.4/sbin/.resolvconf-wrapped: line 1250: kill: (551) - Operation not permitted client # [ 12.621314] dhcpcd[594]: .resolvconf-wrapped: clearing stale lock pid 551 worker1 # [ 12.599518] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 12.601191] dhcpcd[669]: dev: loaded udev worker2 # [ 12.621541] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 12.612340] dhcpcd[636]: eth0: waiting for carrier worker2 # [ 12.615231] dhcpcd[636]: eth0: carrier acquired worker1 # [ 12.611930] network-addresses-eth1-start[655]: adding address 192.168.1.3/24... done worker2 # [ 12.630589] dhcpcd[636]: DUID 00:01:00:01:32:45:19:b2:52:54:00:12:34:56 worker2 # [ 12.631531] dhcpcd[636]: eth0: IAID 00:12:34:56 worker2 # [ 12.634150] dhcpcd[636]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 12.677137] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 12.664798] dhcpcd[543]: eth0: waiting for carrier worker2 # [ 12.651560] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 12.668175] dhcpcd[543]: libudev: received NULL device client # [ 12.668986] dhcpcd[543]: libudev: received NULL device worker1 # [ 12.631284] network-addresses-eth1-start[655]: adding address 2001:db8:1::3/64... done client # [ 12.669818] dhcpcd[543]: eth0: carrier acquired worker2 # [ 12.645423] systemd-logind[503]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 12.656865] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 12.666064] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 12.645973] dhcpcd[644]: eth0: waiting for carrier worker1 # [ 12.646814] dhcpcd[644]: eth0: waiting for carrier worker1 # [ 12.647480] dhcpcd[644]: eth0: carrier acquired worker2 # [ 12.658561] network-addresses-eth1-start[680]: adding address 192.168.1.4/24... done client # [ 12.691701] systemd[1]: Finished Extra networking commands.. client # [ 12.694224] systemd[1]: Reached target Network. client # [ 12.696316] dhcpcd[543]: DUID 00:01:00:01:32:45:19:b2:52:54:00:12:34:56 client # [ 12.698449] dhcpcd[543]: eth0: IAID 00:12:34:56 worker1 # [ 12.663368] systemd[1]: Finished Address configuration of eth1. client # [ 12.699945] dhcpcd[543]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 12.673298] network-addresses-eth1-start[680]: adding address 2001:db8:1::4/64... done worker1 # [ 12.667938] dhcpcd[644]: DUID 00:01:00:01:32:45:19:b2:52:54:00:12:34:56 client # [ 12.703341] systemd[1]: Starting Permit User Sessions... worker1 # [ 12.669888] dhcpcd[644]: eth0: IAID 00:12:34:56 worker1 # [ 12.670562] dhcpcd[644]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.683522] systemd[1]: Finished Extra networking commands.. lb # [ 12.688977] systemd[1]: Reached target Network. worker2 # [ 12.695438] systemd[1]: Finished Address configuration of eth1. lb # [ 12.705455] systemd[1]: Starting PostgreSQL Server... lb # [ 12.713963] systemd[1]: Starting Permit User Sessions... worker1 # [ 12.706099] systemd-logind[520]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 12.752908] systemd[1]: Finished Permit User Sessions. client # [ 12.761282] systemd[1]: Started Getty on tty1. client # [ 12.763905] systemd[1]: Reached target Login Prompts. worker1 # [ 12.805041] mousedev: PS/2 mouse device common for all mice worker2 # [ 12.827293] mousedev: PS/2 mouse device common for all mice lb # [ 12.858166] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 12.848810] systemd[1]: Finished Permit User Sessions. worker1 # [ 12.847785] systemd-logind[520]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 12.859116] systemd[1]: Started Getty on tty1. lb # [ 12.859882] systemd[1]: Reached target Login Prompts. worker2 # [ 12.898632] systemd-logind[503]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 12.922588] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 12.923095] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 12.929052] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 12.929391] cfg80211: failed to load regulatory.db lb # [ 12.927430] dhcpcd[669]: no valid interfaces found lb # [ 12.928943] dhcpcd[669]: no valid interfaces found client: (finished: waiting for unit network-addresses-eth1.service, in 13.35 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } client # [ 12.968523] dhcpcd[543]: eth0: soliciting a DHCP lease client # [ 12.976559] dhcpcd[543]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 12.984266] dhcpcd[543]: eth0: probing address 10.0.2.15/24 client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.06 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 13.058841] (udev-worker)[684]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 13.068823] (udev-worker)[684]: Network interface NamePolicy= disabled on kernel command line. lb # [ 13.091931] (udev-worker)[691]: Network interface NamePolicy= disabled on kernel command line. lb # [ 13.222269] postgresql-pre-start[711]: The files belonging to this database system will be owned by user "postgres". lb # [ 13.223702] postgresql-pre-start[711]: This user must also own the server process. lb # [ 13.240801] postgresql-pre-start[711]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 13.242847] postgresql-pre-start[711]: The default database encoding has accordingly been set to "UTF8". lb # [ 13.247176] postgresql-pre-start[711]: The default text search configuration will be set to "english". lb # [ 13.249426] postgresql-pre-start[711]: Data page checksums are enabled. lb # [ 13.250285] postgresql-pre-start[711]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 13.251520] postgresql-pre-start[711]: creating subdirectories ... ok lb # [ 13.259269] postgresql-pre-start[711]: selecting dynamic shared memory implementation ... posix client # [ 13.394098] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 worker1 # [ 13.387557] dhcpcd[644]: eth0: soliciting a DHCP lease worker1 # [ 13.393050] dhcpcd[644]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 13.400212] dhcpcd[644]: eth0: probing address 10.0.2.15/24 lb # [ 13.446297] postgresql-pre-start[711]: selecting default "max_connections" ... 100 lb # [ 13.473960] systemd-logind[531]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 13.487299] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 lb # [ 13.506286] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 13.521619] systemd[1]: Starting Address configuration of eth1... lb # [ 13.655454] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 13.646140] dhcpcd[669]: eth0: waiting for carrier lb # [ 13.649492] dhcpcd[669]: eth0: carrier acquired lb # [ 13.655281] postgresql-pre-start[711]: selecting default "shared_buffers" ... 128MB lb # [ 13.664839] dhcpcd[669]: DUID 00:01:00:01:32:45:19:b3:52:54:00:12:34:56 lb # [ 13.667223] dhcpcd[669]: eth0: IAID 00:12:34:56 lb # [ 13.669618] dhcpcd[669]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 13.697852] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.685938] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 13.713177] systemd[1]: Starting Virtual Console Setup... lb # [ 13.697492] network-addresses-eth1-start[734]: adding address 192.168.1.2/24... done client # [ 13.737107] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.711081] network-addresses-eth1-start[734]: adding address 2001:db8:1::2/64... done client # [ 13.741743] systemd[1]: Stopped Virtual Console Setup. client # [ 13.749684] systemd[1]: Starting Virtual Console Setup... lb # [ 13.732480] systemd[1]: Finished Address configuration of eth1. lb # [ 13.758378] mousedev: PS/2 mouse device common for all mice client # [ 13.782409] systemd-logind[430]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.773274] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.782648] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.783759] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.788229] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 13.790951] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.794081] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.811770] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.813147] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.815347] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.813962] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.833982] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 13.837340] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.838544] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.894788] systemd-logind[520]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 13.916146] systemd-logind[531]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 13.991000] dhcpcd[636]: eth0: soliciting a DHCP lease worker2 # [ 13.996554] dhcpcd[636]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 14.004219] dhcpcd[636]: eth0: probing address 10.0.2.15/24 worker2 # [ 14.132870] systemd-vconsole-setup[703]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 14.136785] systemd[1]: Finished Virtual Console Setup. client # [ 14.168892] systemd-vconsole-setup[650]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 14.173041] systemd[1]: Finished Virtual Console Setup. client # [ 14.228367] dhcpcd[543]: eth0: soliciting an IPv6 router client # [ 14.230121] dhcpcd[543]: eth0: Router Advertisement from fe80::2 client # [ 14.231244] dhcpcd[543]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 14.232512] dhcpcd[543]: eth0: adding route to fec0::/64 client # [ 14.233471] dhcpcd[543]: eth0: adding default route via fe80::2 worker1 # [ 14.258260] systemd-vconsole-setup[700]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 14.263452] systemd[1]: Finished Virtual Console Setup. lb: (finished: waiting for unit network-addresses-eth1.service, in 1.31 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2 # [ 14.425643] dhcpcd[636]: eth0: soliciting an IPv6 router worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker2 # [ 14.429236] dhcpcd[636]: eth0: Router Advertisement from fe80::2 worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker2 # [ 14.431931] dhcpcd[636]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 14.434813] dhcpcd[636]: eth0: adding route to fec0::/64 worker2 # [ 14.437192] dhcpcd[636]: eth0: adding default route via fe80::2 worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.04 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1 # [ 14.474551] dhcpcd[644]: eth0: soliciting an IPv6 router worker1 # [ 14.475997] dhcpcd[644]: eth0: Router Advertisement from fe80::2 worker1 # [ 14.476977] dhcpcd[644]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 14.477912] dhcpcd[644]: eth0: adding route to fec0::/64 worker1 # [ 14.478666] dhcpcd[644]: eth0: adding default route via fe80::2 worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.11 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.05 seconds) lb: waiting for unit niks3.service lb # [ 14.968021] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 lb # [ 15.467561] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 15.480661] systemd[1]: Starting Virtual Console Setup... lb # [ 15.499416] dhcpcd[669]: eth0: soliciting a DHCP lease lb # [ 15.508357] dhcpcd[669]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 15.514624] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 15.521760] systemd[1]: Stopped Virtual Console Setup. lb # [ 15.522546] dhcpcd[669]: eth0: probing address 10.0.2.15/24 lb # [ 15.536436] systemd[1]: Starting Virtual Console Setup... lb # [ 15.708149] systemd-logind[531]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 16.041123] systemd-vconsole-setup[776]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 16.044753] systemd[1]: Finished Virtual Console Setup. lb # [ 16.077971] dhcpcd[669]: eth0: soliciting an IPv6 router lb # [ 16.079414] dhcpcd[669]: eth0: Router Advertisement from fe80::2 lb # [ 16.084090] dhcpcd[669]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 16.084998] dhcpcd[669]: eth0: adding route to fec0::/64 lb # [ 16.085710] dhcpcd[669]: eth0: adding default route via fe80::2 lb # [ 16.243640] postgresql-pre-start[711]: selecting default time zone ... UTC lb # [ 16.247578] postgresql-pre-start[711]: creating configuration files ... ok lb # [ 16.514840] postgresql-pre-start[711]: running bootstrap script ... ok lb # [ 17.186574] postgresql-pre-start[711]: performing post-bootstrap initialization ... ok lb # [ 17.327772] postgresql-pre-start[711]: syncing data to disk ... ok lb # [ 17.330157] postgresql-pre-start[711]: initdb: warning: enabling "trust" authentication for local connections lb # [ 17.331447] postgresql-pre-start[711]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 17.333602] postgresql-pre-start[711]: Success. You can now start the database server using: lb # [ 17.334780] postgresql-pre-start[711]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 17.467263] postgres[809]: [809] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 17.471658] postgres[809]: [809] LOG: listening on IPv6 address "::1", port 5432 lb # [ 17.473010] postgres[809]: [809] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 17.474766] postgres[809]: [809] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 17.488556] postgres[818]: [818] LOG: database system was shut down at 2026-09-22 10:49:26 GMT lb # [ 17.494532] postgres[809]: [809] LOG: database system is ready to accept connections lb # [ 17.501084] systemd[1]: Started PostgreSQL Server. lb # [ 17.509373] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 17.732829] postgresql-setup-start[829]: CREATE DATABASE lb # [ 17.797867] postgresql-setup-start[835]: CREATE ROLE lb # [ 17.828608] postgresql-setup-start[839]: ALTER DATABASE lb # [ 17.836133] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 17.838074] systemd[1]: Reached target PostgreSQL. worker1 # [ 18.307568] dhcpcd[644]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 18.434489] dhcpcd[543]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 18.311315] dhcpcd[644]: eth0: adding route to 10.0.2.0/24 worker1 # [ 18.312077] dhcpcd[644]: eth0: adding default route via 10.0.2.2 worker1 # [ 18.468910] systemd[1]: Started DHCP Client. worker1 # [ 18.472836] systemd[1]: Reached target Multi-User System. worker1 # [ 18.473732] systemd[1]: Startup finished in 969ms (kernel) + 5.408s (initrd) + 12.094s (userspace) = 18.472s. client # [ 18.438537] dhcpcd[543]: eth0: adding route to 10.0.2.0/24 client # [ 18.438772] dhcpcd[543]: eth0: adding default route via 10.0.2.2 client # [ 18.604249] systemd[1]: Started DHCP Client. client # [ 18.608963] systemd[1]: Reached target Multi-User System. client # [ 18.609788] systemd[1]: Startup finished in 972ms (kernel) + 5.415s (initrd) + 12.220s (userspace) = 18.608s. worker2 # [ 18.684750] dhcpcd[636]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 18.688176] dhcpcd[636]: eth0: adding route to 10.0.2.0/24 worker2 # [ 18.691078] dhcpcd[636]: eth0: adding default route via 10.0.2.2 worker2 # [ 18.855110] systemd[1]: Started DHCP Client. worker2 # [ 18.856959] systemd[1]: Reached target Multi-User System. worker2 # [ 18.858209] systemd[1]: Startup finished in 964ms (kernel) + 4.619s (initrd) + 13.274s (userspace) = 18.857s. lb # [ 20.220391] dhcpcd[669]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 20.223126] dhcpcd[669]: eth0: adding route to 10.0.2.0/24 lb # [ 20.224926] dhcpcd[669]: eth0: adding default route via 10.0.2.2 lb # [ 20.378624] systemd[1]: Started DHCP Client. lb # [ 20.380782] systemd[1]: Reached target Network is Online. lb # [ 20.385684] systemd[1]: Started Envoy reverse proxy. lb # [ 20.394172] systemd[1]: Started mock-oidc.service. lb # [ 20.493464] mock-oidc-server[928]: Mock OIDC Server running lb # [ 20.494785] mock-oidc-server[928]: OIDC Address: 192.168.1.2:8080 lb # [ 20.496100] mock-oidc-server[928]: Issue Address: 0.0.0.0:8081 lb # [ 20.497175] mock-oidc-server[928]: Issuer: http://192.168.1.2:8080/oidc lb # [ 20.498249] mock-oidc-server[928]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 20.499666] mock-oidc-server[928]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 20.501437] mock-oidc-server[928]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 21.073304] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 21.075173] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 21.076726] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 21.079049] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 21.084584] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 21.087689] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 21.091374] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 21.093320] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 21.098295] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 21.102055] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 21.144763] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 21.147465] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 21.150086] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 21.152365] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 21.154688] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 21.162181] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 21.166238] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 21.168478] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 21.170247] envoy[927]: [2026-09-22 10:49:30.654][927][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 21.173384] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 21.175160] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 21.177744] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 21.179878] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 21.188229] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 21.190015] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 21.196491] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 21.199004] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 21.202673] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 21.205158] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 21.207926] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 21.210209] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 21.212095] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 21.217269] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 21.220403] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 21.227677] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 21.230730] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 21.232613] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 21.235145] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 21.237616] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 21.240859] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 21.247781] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 21.249890] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 21.252069] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 21.254007] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 21.258991] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 21.260879] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 21.263733] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 21.265665] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 21.270441] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 21.273842] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 21.276840] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 21.278556] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 21.281160] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 21.294910] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 21.303064] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 21.306160] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 21.308055] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 21.310724] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 21.312948] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 21.317281] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 21.324452] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 21.326300] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 21.334234] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 21.339480] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 21.343052] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 21.345996] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 21.349826] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 21.357243] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 21.360550] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 21.367164] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 21.370610] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 21.377106] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 21.380402] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 21.385713] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 21.399511] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 21.403019] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 21.405647] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 21.407597] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 21.411563] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 21.414209] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 21.416849] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 21.420265] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 21.422875] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 21.425148] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 21.428633] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 21.432051] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 21.433989] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 21.436664] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 21.439125] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 21.443592] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 21.445818] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 21.447658] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 21.451821] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 21.455742] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 21.458499] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 21.460722] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 21.463305] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 21.467876] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 21.470078] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 21.473954] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 21.479883] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 21.482978] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 21.485283] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 21.487259] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 21.490580] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 21.499927] envoy[927]: [2026-09-22 10:49:30.655][927][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 21.502326] envoy[927]: [2026-09-22 10:49:30.722][927][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 21.503972] envoy[927]: [2026-09-22 10:49:30.724][927][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 21.520732] envoy[927]: [2026-09-22 10:49:30.724][927][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 21.522373] envoy[927]: [2026-09-22 10:49:30.724][927][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 21.534490] envoy[927]: [2026-09-22 10:49:30.724][927][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 21.537163] envoy[927]: [2026-09-22 10:49:30.942][927][info][main] [source/server/server.cc:968] runtime: {} lb # [ 21.540098] envoy[927]: [2026-09-22 10:49:30.954][927][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 21.543152] envoy[927]: [2026-09-22 10:49:30.954][927][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 21.545802] envoy[927]: [2026-09-22 10:49:30.954][927][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 21.548205] envoy[927]: [2026-09-22 10:49:30.959][927][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 21.553297] envoy[927]: [2026-09-22 10:49:30.959][927][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 21.554989] envoy[927]: [2026-09-22 10:49:30.959][927][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 21.577739] envoy[927]: [2026-09-22 10:49:31.159][927][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) worker2 # [ 21.579834] systemd[1]: Reached target Network is Online. worker1 # [ 21.575188] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 21.588095] envoy[927]: [2026-09-22 10:49:31.169][927][info][config] [source/server/configuration_impl.cc:164] loading stats configuration worker2 # [ 21.591425] systemd[1]: Starting scheduler-token.service... lb # [ 21.590754] envoy[927]: [2026-09-22 10:49:31.173][927][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 21.594621] envoy[927]: [2026-09-22 10:49:31.173][927][info][main] [source/server/server.cc:1077] starting main dispatch loop worker2 # [ 21.714733] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 21.719364] systemd[1]: Finished scheduler-token.service. worker2 # [ 21.724567] systemd[1]: scheduler-token.service: Consumed 54ms CPU time over 126ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 21.730826] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 21.830596] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 21.843286] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 21.845701] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 21.881275] nix-grpc-daemon[774]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 21.882969] nix-grpc-daemon[774]: E0000 00:00:1790074171.483972 774 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 21.896672] nix-grpc-daemon[774]: E0000 00:00:1790074171.499433 774 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 21.900061] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 21.905093] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 21.938427] nix-grpc-daemon[774]: E0000 00:00:1790074171.540284 815 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 21.944917] envoy[927]: [2026-09-22 10:49:31.527][927][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 21.950344] envoy[927]: [2026-09-22 10:49:31.527][927][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized worker2 # [ 21.957239] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 21.957375] envoy[927]: [2026-09-22 10:49:31.527][927][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager worker1 # [ 21.957928] nix-grpc-daemon[774]: E0000 00:00:1790074171.560599 815 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 21.969319] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 21.966839] envoy[927]: [2026-09-22 10:49:31.549][927][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker2 # [ 21.971831] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 21.964072] nix-grpc-daemon[774]: E0000 00:00:1790074171.563862 815 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 21.980320] nix-grpc-daemon[774]: E0000 00:00:1790074171.582124 815 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 21.995216] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.998352] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.008373] nix-grpc-daemon[780]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 22.010052] nix-grpc-daemon[780]: E0000 00:00:1790074171.611404 780 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 22.023631] nix-grpc-daemon[780]: E0000 00:00:1790074171.626718 780 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 22.027503] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 22.033049] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 22.064190] nix-grpc-daemon[780]: E0000 00:00:1790074171.666586 822 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 22.083889] nix-grpc-daemon[780]: E0000 00:00:1790074171.686888 822 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 22.088066] nix-grpc-daemon[780]: E0000 00:00:1790074171.690234 822 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 22.105798] nix-grpc-daemon[780]: E0000 00:00:1790074171.708191 822 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 22.120900] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.122678] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.199865] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.205340] nix-grpc-daemon[774]: ts=2026-09-22T10:49:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.324205] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.328291] nix-grpc-daemon[780]: ts=2026-09-22T10:49:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.406418] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.411594] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.529043] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.534365] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.612452] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.615675] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.735539] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.741053] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.816855] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.821968] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.942245] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.947578] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.024782] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.028900] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.150615] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.154780] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.225369] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.230486] nix-grpc-daemon[774]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.351222] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.356453] nix-grpc-daemon[780]: ts=2026-09-22T10:49:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.431439] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.436713] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.557563] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.562635] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.637469] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.642551] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.763791] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.769157] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.843666] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.848820] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.970519] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.975667] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.051428] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.055885] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.178668] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.182869] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.252202] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.257314] nix-grpc-daemon[774]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.380051] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.385239] nix-grpc-daemon[780]: ts=2026-09-22T10:49:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.458296] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.463469] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.586528] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.591819] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.664519] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.669664] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.793234] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.798432] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.870462] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.875642] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.999742] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.005308] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.078566] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.082840] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.208430] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.212578] nix-grpc-daemon[780]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.280857] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.286593] nix-grpc-daemon[774]: ts=2026-09-22T10:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.409498] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.414888] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.487730] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.493305] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 25.594590] rustfs-bucket-start[981]: mb s3://farm lb # [ 25.603311] systemd[1]: Finished rustfs-bucket.service. lb # [ 25.611783] systemd[1]: Starting niks3 server... worker2 # [ 25.616710] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.622484] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.693926] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.699283] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.823157] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.828500] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 25.855725] postgres[998]: [998] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 25.858021] postgres[998]: [998] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 25.891823] niks3-server[991]: 2026/09/22 10:49:35 OK 20241026095416_initial_model.sql (21.81ms) lb # [ 25.895534] niks3-server[991]: 2026/09/22 10:49:35 OK 20251210153512_drop_unused_gin_index.sql (1.95ms) lb # [ 25.901823] niks3-server[991]: 2026/09/22 10:49:35 OK 20251218171726_add_pins.sql (6.25ms) lb # [ 25.906550] niks3-server[991]: 2026/09/22 10:49:35 OK 20260628120000_add_object_size_and_stats.sql (4.74ms) worker1 # [ 25.900398] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 lb # [ 25.912446] niks3-server[991]: 2026/09/22 10:49:35 OK 20260905000000_add_claims.sql (4.39ms) lb # [ 25.917150] niks3-server[991]: 2026/09/22 10:49:35 OK 20260920000000_drop_claims.sql (4.56ms) worker1 # [ 25.905071] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 25.918423] niks3-server[991]: 2026/09/22 10:49:35 goose: successfully migrated database to version: 20260920000000 lb # [ 25.923885] niks3-server[991]: 2026/09/22 10:49:35 OK 1_commit_pending_closure.sql (6.71ms) lb # [ 25.927056] niks3-server[991]: 2026/09/22 10:49:35 OK 2_object_stats_trigger.sql (3.01ms) lb # [ 25.928636] niks3-server[991]: 2026/09/22 10:49:35 goose: up to current file version: 2 lb # [ 25.932488] niks3-server[991]: 2026/09/22 10:49:35 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key lb # [ 25.979747] niks3-server[991]: 2026/09/22 10:49:35 INFO Read proxy enabled — serving cache objects from S3 lb # [ 25.982613] niks3-server[991]: 2026/09/22 10:49:35 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 25.986707] niks3-server[991]: 2026/09/22 10:49:35 INFO systemd watchdog enabled interval=15s lb # [ 25.993462] niks3-server[991]: 2026/09/22 10:49:35 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 25.995712] systemd[1]: Started niks3 server. lb # [ 26.001159] systemd[1]: Reached target Multi-User System. lb # [ 26.004316] systemd[1]: Startup finished in 986ms (kernel) + 5.571s (initrd) + 19.430s (userspace) = 25.989s. worker2 # [ 26.030366] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.035623] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 11.47 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 26.107379] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.111667] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.06 seconds) lb: waiting for TCP port 50051 on localhost worker2 # [ 26.238545] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.242900] nix-grpc-daemon[780]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 26.272891] envoy[927]: conn peer=[::1]:53452 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=3 lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds) worker1 # [ 26.311405] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.316265] nix-grpc-daemon[774]: ts=2026-09-22T10:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.439324] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.446091] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.476863] systemd[1]: Starting scheduler-token.service... worker1 # [ 26.510264] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.515518] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.569910] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 26.574944] systemd[1]: Finished scheduler-token.service. worker2 # [ 26.576673] systemd[1]: scheduler-token.service: Consumed 40ms CPU time over 97ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 26.646974] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.651409] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 26.716545] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.721647] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2 # [ 26.852658] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.857772] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.922423] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.927346] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.058773] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.063936] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.130099] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.134228] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.267261] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.271595] nix-grpc-daemon[780]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.330581] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.335732] nix-grpc-daemon[774]: ts=2026-09-22T10:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.468124] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.473499] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.536806] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.541961] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.675082] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.681036] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 27.742681] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.747873] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 27.882369] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.887963] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.948646] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.953697] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.089086] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.094517] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.156510] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.160576] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.297064] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.301907] nix-grpc-daemon[780]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.357447] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.363092] nix-grpc-daemon[774]: ts=2026-09-22T10:49:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.503038] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.508266] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.563874] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.569081] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.709326] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.714508] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.769912] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.775119] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 28.915792] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.924768] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.975809] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.981165] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.125941] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.131262] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.185213] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.194698] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.334335] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.338554] nix-grpc-daemon[780]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.384402] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.389578] nix-grpc-daemon[774]: ts=2026-09-22T10:49:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.535434] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.540891] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.590496] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.595724] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.741908] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.747434] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.796685] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.801988] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 29.948602] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.953958] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 30.002817] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.008674] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.155334] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.160753] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.211462] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.215845] nix-grpc-daemon[774]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.363872] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.368352] nix-grpc-daemon[780]: ts=2026-09-22T10:49:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.412103] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.417526] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.565100] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.570445] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.618338] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.623659] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.771472] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.776752] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.824788] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.829925] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.977775] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.982853] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 31.030653] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.035983] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 31.183879] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.189205] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.238555] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.242707] nix-grpc-daemon[774]: ts=2026-09-22T10:49:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.393483] nix-grpc-daemon[780]: ts=2026-09-22T10:49:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.398618] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.439916] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.445492] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.600321] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.605763] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.643293] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.649075] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.807094] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.813181] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.846198] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.852620] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.014822] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.020563] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.053856] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 32.059986] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker2 # [ 32.221972] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.228435] nix-grpc-daemon[780]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.263999] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.268412] nix-grpc-daemon[774]: ts=2026-09-22T10:49:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.430694] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.434047] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.464522] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.470045] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.632364] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.637503] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.670714] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.675083] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.838647] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.844147] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.877476] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.881963] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.045966] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.051571] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.078826] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.084767] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 33.253706] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.259915] nix-grpc-daemon[780]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker1 # [ 33.289284] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.293562] nix-grpc-daemon[774]: ts=2026-09-22T10:49:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.463875] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.468172] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.489672] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.495415] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.664773] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.670347] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.696682] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.702217] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.872085] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.877755] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.903099] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.908928] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.080470] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.085683] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.110196] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.115942] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 34.287148] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.293436] nix-grpc-daemon[780]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.318611] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.324104] nix-grpc-daemon[774]: ts=2026-09-22T10:49:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) worker2 # [ 34.497483] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.501901] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.525953] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.531661] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.698890] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.705102] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.732730] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.738307] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.906817] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.912676] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.939788] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.945474] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.113994] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.119773] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.146312] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.151797] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.321045] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.326624] nix-grpc-daemon[780]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.355746] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 35.360620] nix-grpc-daemon[774]: ts=2026-09-22T10:49:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker2 # [ 35.531145] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.535521] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.556622] nix-grpc-daemon[774]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.562013] nix-grpc-daemon[774]: ts=2026-09-22T10:49:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.733102] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.736403] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.763176] nix-grpc-daemon[774]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.769331] nix-grpc-daemon[774]: ts=2026-09-22T10:49:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.938273] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.943904] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.969899] nix-grpc-daemon[774]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.973040] nix-grpc-daemon[774]: ts=2026-09-22T10:49:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.998126] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_take_over lb # [ 36.003614] niks3-server[991]: 2026/09/22 10:49:45 INFO lead: acquired remote=192.168.1.4:35718 worker2 # [ 36.145046] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.176204] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 36.174028] nix-grpc-daemon[774]: ts=2026-09-22T10:49:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.181639] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45454 worker2 # [ 36.188809] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 36.194487] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45454 worker2 # [ 36.199417] nix-grpc-daemon[780]: ts=2026-09-22T10:49:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.11 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.89 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 37.987625] nix-grpc-daemon[774]: ts=2026-09-22T10:49:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 38.009936] systemd[1]: Started Nix Daemon. worker1 # [ 38.349875] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 38.377790] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5) worker1 # [ 38.385811] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5) worker1 # [ 38.453042] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 243 ms (attempt 3/5) worker1 # [ 38.696625] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 4/5) worker1 # [ 38.796441] nix-grpc-daemon[774]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 38.802611] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 38.823308] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 38.841558] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=debug event=path_info_miss path=5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test error="error: path '/nix/store/5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 38.854902] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 38.867756] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=debug event=path_info_miss path=5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test error="error: path '/nix/store/5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds) client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds) worker1 # [ 39.360432] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 39.366661] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:34278 worker1 # [ 39.375422] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:34278 worker1 # [ 39.383267] nix-grpc-daemon[774]: ts=2026-09-22T10:49:48Z level=debug event=path_info_miss path=5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test error="error: path '/nix/store/5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 39.420849] nix-grpc-daemon[780]: ts=2026-09-22T10:49:49Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:45466 worker2 # [ 39.441721] systemd[1]: Started Nix Daemon. worker2 # [ 39.765854] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 39.790510] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5) worker2 # [ 39.826626] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 2/5) worker2 # [ 39.911007] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 346 ms (attempt 3/5) worker2 # [ 40.257515] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 4/5) worker2 # [ 40.439562] nix-grpc-daemon[780]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 40.444827] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 40.458121] nix-grpc-daemon[780]: ts=2026-09-22T10:49:50Z level=debug event=path_info_miss path=5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test error="error: path '/nix/store/5252da3xcqz5fxzqz01xd82cxl855ak1-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds) worker1 # [ 40.800697] nix-grpc-daemon[774]: ts=2026-09-22T10:49:50Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 41.074377] nix-grpc-daemon[780]: ts=2026-09-22T10:49:50Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.54 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2 worker2 # [ 41.343207] nix-grpc-daemon[780]: ts=2026-09-22T10:49:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45466 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 41.348238] nix-grpc-daemon[774]: ts=2026-09-22T10:49:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker2 # [ 41.398636] nix-grpc-daemon[780]: ts=2026-09-22T10:49:51Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker2 # [ 41.406330] nix-grpc-daemon[780]: ts=2026-09-22T10:49:51Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 41.453431] nix-grpc-daemon[780]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 41.473886] nix-grpc-daemon[780]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying 1 paths to 'lb:50051'... worker2 # [ 41.490407] nix-grpc-daemon[780]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker2 # [ 41.522844] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 41.559970] nix-grpc-daemon[780]: ts=2026-09-22T10:49:51Z level=info event=niks3_push_started pid=880 lb # [ 41.575295] envoy[927]: conn peer=[2001:db8:1::1]:48890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=365 ms=947 lb # [ 41.580828] envoy[927]: conn peer=[2001:db8:1::1]:48906 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2630 tx=422 ms=1122 lb # [ 41.585548] envoy[927]: conn peer=[2001:db8:1::1]:43760 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=12 lb # [ 41.590087] envoy[927]: conn peer=[2001:db8:1::1]:43762 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=19 worker2 # [ 41.851764] nix-grpc-daemon[880]: time=2026-09-22T10:49:51.453Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 42.048888] nix-daemon[866]: accepted connection from pid 893, user nix-grpc-daemon (trusted) worker2 # [ 42.069711] nix-daemon[866]: reaped child process 896, status = succeeded worker2 # [ 42.257576] nix-daemon[866]: accepted connection from pid 898, user nix-grpc-daemon (trusted) worker2 # [ 42.279091] nix-daemon[866]: reaped child process 901, status = succeeded lb # [ 42.301531] niks3-server[991]: 2026/09/22 10:49:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 42.328217] nix-grpc-daemon[880]: time=2026-09-22T10:49:51.931Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 42.330867] nix-grpc-daemon[880]: time=2026-09-22T10:49:51.933Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 42.380886] niks3-server[991]: 2026/09/22 10:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 42.391403] niks3-server[991]: 2026/09/22 10:49:51 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 42.418515] niks3-server[991]: 2026/09/22 10:49:52 INFO Signed narinfos id=1 count=1 worker2 # [ 42.423215] nix-grpc-daemon[880]: time=2026-09-22T10:49:52.026Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.438368] niks3-server[991]: 2026/09/22 10:49:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 42.445268] niks3-server[991]: 2026/09/22 10:49:52 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 42.452554] niks3-server[991]: 2026/09/22 10:49:52 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo lb # [ 42.456355] niks3-server[991]: 2026/09/22 10:49:52 INFO Completed upload id=1 worker2 # [ 42.457236] nix-grpc-daemon[880]: time=2026-09-22T10:49:52.060Z level=INFO msg="Upload complete. (598ms)" worker2 # [ 42.469452] nix-grpc-daemon[780]: ts=2026-09-22T10:49:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 42.487626] nix-grpc-daemon[780]: ts=2026-09-22T10:49:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 42.498287] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 42.506032] nix-daemon[866]: reaped child process 878, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker2 # [ 42.514544] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker2 # [ 44.138086] nix-daemon[866]: reaped child process 906, status = succeeded worker2 # [ 44.301807] nix-daemon[866]: accepted connection from pid 922, user nix-grpc-daemon (trusted) worker2 # [ 44.318706] nix-daemon[866]: reaped child process 925, status = succeeded lb # [ 44.330349] niks3-server[991]: 2026/09/22 10:49:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 44.344399] nix-grpc-daemon[880]: time=2026-09-22T10:49:53.946Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 44.345865] nix-grpc-daemon[880]: time=2026-09-22T10:49:53.946Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 44.378166] niks3-server[991]: 2026/09/22 10:49:53 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 44.391220] niks3-server[991]: 2026/09/22 10:49:53 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 44.403816] niks3-server[991]: 2026/09/22 10:49:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign worker2 # [ 44.411861] nix-grpc-daemon[880]: time=2026-09-22T10:49:54.013Z level=INFO msg="Uploading 1 narinfos" lb # [ 44.412648] niks3-server[991]: 2026/09/22 10:49:53 INFO Signed narinfos id=2 count=1 lb # [ 44.416858] niks3-server[991]: 2026/09/22 10:49:53 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 44.430039] niks3-server[991]: 2026/09/22 10:49:54 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 44.439881] niks3-server[991]: 2026/09/22 10:49:54 INFO Completed upload id=2 worker2 # [ 44.440537] nix-grpc-daemon[880]: time=2026-09-22T10:49:54.043Z level=INFO msg="Upload complete. (312ms)" lb # [ 44.443139] niks3-server[991]: 2026/09/22 10:49:54 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker2 # [ 44.464715] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker2 # [ 44.478092] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker2 # [ 44.481626] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 44.485298] nix-daemon[866]: reaped child process 904, status = succeeded worker2 # [ 44.509047] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 44.524447] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 44.539764] nix-grpc-daemon[880]: time=2026-09-22T10:49:54.142Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 44.550025] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker2 # [ 44.574401] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 44.782543] nix-daemon[866]: accepted connection from pid 931, user nix-grpc-daemon (trusted) worker2 # [ 44.799969] nix-daemon[866]: reaped child process 934, status = succeeded worker2 # [ 44.971918] nix-daemon[866]: accepted connection from pid 936, user nix-grpc-daemon (trusted) worker2 # [ 44.988665] nix-daemon[866]: reaped child process 939, status = succeeded lb # [ 44.999769] niks3-server[991]: 2026/09/22 10:49:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 45.011250] nix-grpc-daemon[880]: time=2026-09-22T10:49:54.614Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 45.012875] nix-grpc-daemon[880]: time=2026-09-22T10:49:54.614Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 45.028450] niks3-server[991]: 2026/09/22 10:49:54 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 45.042450] niks3-server[991]: 2026/09/22 10:49:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign worker2 # [ 45.048966] nix-grpc-daemon[880]: time=2026-09-22T10:49:54.652Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.054049] niks3-server[991]: 2026/09/22 10:49:54 INFO Signed narinfos id=3 count=1 lb # [ 45.061313] niks3-server[991]: 2026/09/22 10:49:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 45.069575] niks3-server[991]: 2026/09/22 10:49:54 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 45.079401] niks3-server[991]: 2026/09/22 10:49:54 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo lb # [ 45.083762] niks3-server[991]: 2026/09/22 10:49:54 INFO Completed upload id=3 worker2 # [ 45.084428] nix-grpc-daemon[880]: time=2026-09-22T10:49:54.687Z level=INFO msg="Upload complete. (492ms)" worker2 # [ 45.093208] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 45.107099] nix-grpc-daemon[780]: ts=2026-09-22T10:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 45.113986] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 45.118583] nix-daemon[866]: reaped child process 929, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker2 # [ 45.132118] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker2 # [ 45.966981] nix-daemon[866]: reaped child process 943, status = succeeded worker2 # [ 46.134156] nix-daemon[866]: accepted connection from pid 960, user nix-grpc-daemon (trusted) worker2 # [ 46.153676] nix-daemon[866]: reaped child process 963, status = succeeded lb # [ 46.165516] niks3-server[991]: 2026/09/22 10:49:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 46.172430] nix-grpc-daemon[880]: time=2026-09-22T10:49:55.774Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 46.173921] nix-grpc-daemon[880]: time=2026-09-22T10:49:55.774Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 46.190174] niks3-server[991]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 46.194546] niks3-server[991]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst lb # [ 46.197822] niks3-server[991]: 2026/09/22 10:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign worker2 # [ 46.200635] nix-grpc-daemon[880]: time=2026-09-22T10:49:55.803Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.202606] niks3-server[991]: 2026/09/22 10:49:55 INFO Signed narinfos id=4 count=1 lb # [ 46.205445] niks3-server[991]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 46.209471] niks3-server[991]: 2026/09/22 10:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 46.213268] niks3-server[991]: 2026/09/22 10:49:55 INFO Completed upload id=4 worker2 # [ 46.213875] nix-grpc-daemon[880]: time=2026-09-22T10:49:55.817Z level=INFO msg="Upload complete. (271ms)" lb # [ 46.214614] niks3-server[991]: 2026/09/22 10:49:55 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker2 # [ 46.223830] nix-grpc-daemon[780]: ts=2026-09-22T10:49:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker2 # [ 46.234166] nix-grpc-daemon[780]: ts=2026-09-22T10:49:55Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker2 # [ 46.236524] nix-grpc-daemon[780]: ts=2026-09-22T10:49:55Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 46.239073] nix-daemon[866]: reaped child process 941, status = succeeded worker2 # [ 46.267546] nix-grpc-daemon[780]: ts=2026-09-22T10:49:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 46.278122] nix-grpc-daemon[780]: ts=2026-09-22T10:49:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 46.287797] nix-grpc-daemon[880]: time=2026-09-22T10:49:55.891Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker2 # [ 46.295937] nix-grpc-daemon[780]: ts=2026-09-22T10:49:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 46.312360] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 46.497911] nix-daemon[866]: accepted connection from pid 969, user nix-grpc-daemon (trusted) worker2 # [ 46.517436] nix-daemon[866]: reaped child process 972, status = succeeded worker2 # [ 46.691799] nix-daemon[866]: accepted connection from pid 974, user nix-grpc-daemon (trusted) worker2 # [ 46.709778] nix-daemon[866]: reaped child process 977, status = succeeded lb # [ 46.721401] niks3-server[991]: 2026/09/22 10:49:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 46.732140] nix-grpc-daemon[880]: time=2026-09-22T10:49:56.335Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 46.734070] nix-grpc-daemon[880]: time=2026-09-22T10:49:56.336Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 46.751368] niks3-server[991]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 46.765984] niks3-server[991]: 2026/09/22 10:49:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign worker2 # [ 46.773351] nix-grpc-daemon[880]: time=2026-09-22T10:49:56.376Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.777667] niks3-server[991]: 2026/09/22 10:49:56 INFO Signed narinfos id=5 count=1 lb # [ 46.783912] niks3-server[991]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 46.793403] niks3-server[991]: 2026/09/22 10:49:56 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 46.800616] niks3-server[991]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo lb # [ 46.805040] niks3-server[991]: 2026/09/22 10:49:56 INFO Completed upload id=5 worker2 # [ 46.805786] nix-grpc-daemon[880]: time=2026-09-22T10:49:56.408Z level=INFO msg="Upload complete. (480ms)" worker2 # [ 46.816340] nix-grpc-daemon[780]: ts=2026-09-22T10:49:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 46.831905] nix-grpc-daemon[780]: ts=2026-09-22T10:49:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 46.838131] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 46.842706] nix-daemon[866]: reaped child process 967, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... worker2 # [ 46.853986] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker2 # [ 48.498362] nix-daemon[866]: reaped child process 981, status = succeeded worker2 # [ 48.672605] nix-daemon[866]: accepted connection from pid 997, user nix-grpc-daemon (trusted) worker2 # [ 48.689523] nix-daemon[866]: reaped child process 1000, status = succeeded lb # [ 48.703269] niks3-server[991]: 2026/09/22 10:49:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 48.717435] nix-grpc-daemon[880]: time=2026-09-22T10:49:58.318Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 48.718917] nix-grpc-daemon[880]: time=2026-09-22T10:49:58.318Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 48.738274] niks3-server[991]: 2026/09/22 10:49:58 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 48.752253] niks3-server[991]: 2026/09/22 10:49:58 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 48.765684] niks3-server[991]: 2026/09/22 10:49:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker2 # [ 48.770545] nix-grpc-daemon[880]: time=2026-09-22T10:49:58.373Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.774261] niks3-server[991]: 2026/09/22 10:49:58 INFO Signed narinfos id=6 count=1 lb # [ 48.779639] niks3-server[991]: 2026/09/22 10:49:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 48.787440] niks3-server[991]: 2026/09/22 10:49:58 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete worker2 # [ 48.793507] nix-grpc-daemon[880]: time=2026-09-22T10:49:58.396Z level=INFO msg="Upload complete. (326ms)" lb # [ 48.795545] niks3-server[991]: 2026/09/22 10:49:58 INFO Completed upload id=6 lb # [ 48.799445] niks3-server[991]: 2026/09/22 10:49:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker2 # [ 48.806545] nix-grpc-daemon[780]: ts=2026-09-22T10:49:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker2 # [ 48.823261] nix-daemon[866]: reaped child process 979, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.75 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.40 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2 worker2 # [ 50.340952] nix-grpc-daemon[780]: ts=2026-09-22T10:49:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45466 client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked): client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # [ 50.353323] nix-grpc-daemon[774]: ts=2026-09-22T10:49:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker2 # [ 50.388870] nix-grpc-daemon[780]: ts=2026-09-22T10:49:59Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.34 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.45 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 51.423883] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45466 worker1 # [ 51.419761] nix-grpc-daemon[774]: ts=2026-09-22T10:50:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker2 # [ 51.444116] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker2 # [ 51.450368] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 51.496784] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 51.512179] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying 1 paths to 'lb:50051'... worker2 # [ 51.528551] nix-grpc-daemon[880]: time=2026-09-22T10:50:01.130Z level=INFO msg="All 2 paths already cached" client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker2 # [ 51.533738] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 51.548563] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) lb # [ 51.580538] envoy[927]: conn peer=[2001:db8:1::1]:43776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4104 ms=7493 worker2 # [ 51.750930] nix-daemon[866]: accepted connection from pid 1029, user nix-grpc-daemon (trusted) worker2 # [ 51.767996] nix-daemon[866]: reaped child process 1032, status = succeeded worker2 # [ 51.941168] nix-daemon[866]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 51.957744] nix-daemon[866]: reaped child process 1037, status = succeeded lb # [ 51.585531] envoy[927]: conn peer=[2001:db8:1::1]:43790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=69 lb # [ 51.969973] niks3-server[991]: 2026/09/22 10:50:01 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 51.983971] nix-grpc-daemon[880]: time=2026-09-22T10:50:01.586Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 51.985722] nix-grpc-daemon[880]: time=2026-09-22T10:50:01.586Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 52.004127] niks3-server[991]: 2026/09/22 10:50:01 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst lb # [ 52.016311] niks3-server[991]: 2026/09/22 10:50:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker2 # [ 52.024243] nix-grpc-daemon[880]: time=2026-09-22T10:50:01.627Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.023701] niks3-server[991]: 2026/09/22 10:50:01 INFO Signed narinfos id=7 count=1 lb # [ 52.030485] niks3-server[991]: 2026/09/22 10:50:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 52.044772] niks3-server[991]: 2026/09/22 10:50:01 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete worker2 # [ 52.051882] nix-grpc-daemon[880]: time=2026-09-22T10:50:01.655Z level=INFO msg="Upload complete. (485ms)" lb # [ 52.052075] niks3-server[991]: 2026/09/22 10:50:01 INFO Completed upload id=7 lb # [ 52.056100] niks3-server[991]: 2026/09/22 10:50:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker2 # [ 52.060286] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 52.075229] nix-grpc-daemon[780]: ts=2026-09-22T10:50:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 52.081622] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 52.086901] nix-daemon[866]: reaped child process 1027, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 52.098876] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 52.720878] nix-daemon[866]: reaped child process 1041, status = succeeded client # farm-top> LOG-farm-top-t1b worker2 # [ 52.874515] nix-daemon[866]: accepted connection from pid 1057, user nix-grpc-daemon (trusted) worker2 # [ 52.891914] nix-daemon[866]: reaped child process 1060, status = succeeded lb # [ 52.906146] niks3-server[991]: 2026/09/22 10:50:02 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 52.918839] nix-grpc-daemon[880]: time=2026-09-22T10:50:02.519Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 52.920393] nix-grpc-daemon[880]: time=2026-09-22T10:50:02.519Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 52.940104] niks3-server[991]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 52.955687] niks3-server[991]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv lb # [ 52.965251] niks3-server[991]: 2026/09/22 10:50:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker2 # [ 52.972194] nix-grpc-daemon[880]: time=2026-09-22T10:50:02.575Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.974446] niks3-server[991]: 2026/09/22 10:50:02 INFO Signed narinfos id=8 count=1 lb # [ 52.980410] niks3-server[991]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 52.991057] niks3-server[991]: 2026/09/22 10:50:02 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 52.996288] niks3-server[991]: 2026/09/22 10:50:02 INFO Completed upload id=8 worker2 # [ 52.996915] nix-grpc-daemon[880]: time=2026-09-22T10:50:02.600Z level=INFO msg="Upload complete. (287ms)" lb # [ 52.999084] niks3-server[991]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker2 # [ 53.009004] nix-grpc-daemon[780]: ts=2026-09-22T10:50:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker2 # [ 53.023421] nix-daemon[866]: reaped child process 1039, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.89 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.53 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup client # [ 53.590746] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup. client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup worker2 # [ 53.829447] nix-grpc-daemon[780]: ts=2026-09-22T10:50:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45466 client # Running as unit: dedup1.service; invocation ID: 2ca2d044966f41369129059539af8f45 client # [ 53.865218] nix[881]: don't know how to build these paths: client # [ 53.867135] nix[881]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 53.850557] nix-grpc-daemon[780]: ts=2026-09-22T10:50:03Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 53.856714] nix-grpc-daemon[780]: ts=2026-09-22T10:50:03Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 53.890548] nix-grpc-daemon[774]: ts=2026-09-22T10:50:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 53.908417] nix-grpc-daemon[774]: ts=2026-09-22T10:50:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 53.923275] nix-grpc-daemon[774]: ts=2026-09-22T10:50:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 client # [ 53.963797] nix[881]: copying 1 paths to 'lb:50051'... client # [ 53.966136] nix[881]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 53.945871] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 53.976133] nix-grpc-daemon[774]: ts=2026-09-22T10:50:03Z level=info event=niks3_push_started pid=867 worker1 # [ 54.244854] nix-grpc-daemon[867]: time=2026-09-22T10:50:03.846Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca="" worker1 # [ 54.440125] nix-daemon[839]: accepted connection from pid 880, user nix-grpc-daemon (trusted) worker1 # [ 54.457738] nix-daemon[839]: reaped child process 883, status = succeeded worker1 # [ 54.630076] nix-daemon[839]: accepted connection from pid 885, user nix-grpc-daemon (trusted) worker1 # [ 54.645551] nix-daemon[839]: reaped child process 888, status = succeeded lb # [ 54.666343] niks3-server[991]: 2026/09/22 10:50:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 54.667521] nix-grpc-daemon[867]: time=2026-09-22T10:50:04.270Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 54.670889] nix-grpc-daemon[867]: time=2026-09-22T10:50:04.272Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" client # [ 54.710726] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 271fa0e3bb11494085c21e6e85abdb5b client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds) worker1 # [ 54.710736] nix-grpc-daemon[867]: time=2026-09-22T10:50:04.313Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.710095] niks3-server[991]: 2026/09/22 10:50:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 54.721680] niks3-server[991]: 2026/09/22 10:50:04 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 54.732621] niks3-server[991]: 2026/09/22 10:50:04 INFO Signed narinfos id=9 count=1 lb # [ 54.734973] niks3-server[991]: 2026/09/22 10:50:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 54.743260] niks3-server[991]: 2026/09/22 10:50:04 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 54.752221] niks3-server[991]: 2026/09/22 10:50:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo lb # [ 54.757095] niks3-server[991]: 2026/09/22 10:50:04 INFO Completed upload id=9 worker1 # [ 54.749498] nix-grpc-daemon[867]: time=2026-09-22T10:50:04.352Z level=INFO msg="Upload complete. (496ms)" worker1 # [ 54.753929] nix-grpc-daemon[774]: ts=2026-09-22T10:50:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 54.763195] nix-grpc-daemon[774]: ts=2026-09-22T10:50:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 54.768681] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 54.771522] nix-daemon[839]: reaped child process 865, status = succeeded client # [ 54.818881] nix[881]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 54.784238] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 55.049338] nix-grpc-daemon[774]: ts=2026-09-22T10:50:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34278 client # [ 55.099623] nix[897]: this derivation will be built: worker2 # [ 55.076615] nix-grpc-daemon[780]: ts=2026-09-22T10:50:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 55.095593] nix-grpc-daemon[780]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' from 'http://lb:5751'... client # [ 55.145753] nix[897]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 55.133260] nix-grpc-daemon[780]: ts=2026-09-22T10:50:04Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 55.133682] nix-grpc-daemon[774]: ts=2026-09-22T10:50:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 client # [ 55.177255] nix[897]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.07 seconds) worker1 # [ 59.412442] nix-daemon[839]: reaped child process 896, status = succeeded client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 59.573593] nix-daemon[839]: accepted connection from pid 926, user nix-grpc-daemon (trusted) worker1 # [ 59.590442] nix-daemon[839]: reaped child process 929, status = succeeded lb # [ 59.611788] niks3-server[991]: 2026/09/22 10:50:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 59.618233] nix-grpc-daemon[867]: time=2026-09-22T10:50:09.219Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 59.620106] nix-grpc-daemon[867]: time=2026-09-22T10:50:09.219Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 59.656111] niks3-server[991]: 2026/09/22 10:50:09 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 59.677605] niks3-server[991]: 2026/09/22 10:50:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign worker1 # [ 59.678209] nix-grpc-daemon[867]: time=2026-09-22T10:50:09.279Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.688985] niks3-server[991]: 2026/09/22 10:50:09 INFO Signed narinfos id=10 count=1 lb # [ 59.694099] niks3-server[991]: 2026/09/22 10:50:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 59.704358] niks3-server[991]: 2026/09/22 10:50:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 59.712491] niks3-server[991]: 2026/09/22 10:50:09 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 59.718881] niks3-server[991]: 2026/09/22 10:50:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo lb # [ 59.724793] niks3-server[991]: 2026/09/22 10:50:09 INFO Completed upload id=10 worker1 # [ 59.716731] nix-grpc-daemon[867]: time=2026-09-22T10:50:09.319Z level=INFO msg="Upload complete. (319ms)" worker1 # [ 59.729198] nix-grpc-daemon[774]: ts=2026-09-22T10:50:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 worker1 # [ 59.746907] nix-grpc-daemon[774]: ts=2026-09-22T10:50:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 worker1 # [ 59.754003] nix-daemon[839]: reaped child process 894, status = succeeded client # [ 59.807694] systemd[1]: dedup2.service: Deactivated successfully. client # [ 59.813735] systemd[1]: dedup2.service: Consumed 225ms CPU time over 5.096s wall clock time, 33.2M memory peak, 6.5K incoming IP traffic, 5.8K outgoing IP traffic. client # [ 59.857544] systemd[1]: dedup1.service: Deactivated successfully. client # [ 59.859779] systemd[1]: dedup1.service: Consumed 177ms CPU time over 6.269s wall clock time, 41.3M memory peak, 7.2K incoming IP traffic, 8.5K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: two clients wanting the same drv build it once, in 7.45 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share client # [ 61.030864] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share. client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share client # Running as unit: share1.service; invocation ID: 350a011f71db4cbaa2916fe0152fbed1 client # [ 61.302125] nix[936]: don't know how to build these paths: client # [ 61.303903] nix[936]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 61.273083] nix-grpc-daemon[780]: ts=2026-09-22T10:50:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 61.287761] nix-grpc-daemon[780]: ts=2026-09-22T10:50:10Z level=debug event=want client=6 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 61.295100] nix-grpc-daemon[780]: ts=2026-09-22T10:50:10Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 61.333266] nix-grpc-daemon[780]: ts=2026-09-22T10:50:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 61.349476] nix-grpc-daemon[780]: ts=2026-09-22T10:50:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 client # [ 61.392234] nix[936]: copying 1 paths to 'lb:50051'... worker2 # [ 61.364684] nix-grpc-daemon[780]: ts=2026-09-22T10:50:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 client # [ 61.394544] nix[936]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 61.381889] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) lb # [ 61.581609] envoy[927]: conn peer=[2001:db8:1::1]:56508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4142 tx=1791 ms=1597 lb # [ 61.586598] envoy[927]: conn peer=[2001:db8:1::1]:56522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1487 ms=4693 worker2 # [ 61.591052] nix-daemon[866]: accepted connection from pid 1101, user nix-grpc-daemon (trusted) lb # [ 61.591573] envoy[927]: conn peer=[2001:db8:1::1]:56512 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1451 ms=5968 worker2 # [ 61.611418] nix-daemon[866]: reaped child process 1104, status = succeeded worker2 # [ 61.782167] nix-daemon[866]: accepted connection from pid 1106, user nix-grpc-daemon (trusted) worker2 # [ 61.799120] nix-daemon[866]: reaped child process 1109, status = succeeded lb # [ 61.810799] niks3-server[991]: 2026/09/22 10:50:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 61.820988] nix-grpc-daemon[880]: time=2026-09-22T10:50:11.423Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 61.822446] nix-grpc-daemon[880]: time=2026-09-22T10:50:11.423Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 61.838702] niks3-server[991]: 2026/09/22 10:50:11 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst worker2 # [ 61.862697] nix-grpc-daemon[880]: time=2026-09-22T10:50:11.465Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.853301] niks3-server[991]: 2026/09/22 10:50:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 61.867006] niks3-server[991]: 2026/09/22 10:50:11 INFO Signed narinfos id=11 count=1 lb # [ 61.873860] niks3-server[991]: 2026/09/22 10:50:11 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls worker2 # [ 61.885493] nix-grpc-daemon[880]: time=2026-09-22T10:50:11.488Z level=INFO msg="Upload complete. (486ms)" lb # [ 61.886571] niks3-server[991]: 2026/09/22 10:50:11 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 61.892326] niks3-server[991]: 2026/09/22 10:50:11 INFO Completed upload id=11 worker2 # [ 61.892368] nix-grpc-daemon[780]: ts=2026-09-22T10:50:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=1 nar_bytes_in=432 lb # [ 61.895801] niks3-server[991]: 2026/09/22 10:50:11 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 61.900659] nix-grpc-daemon[780]: ts=2026-09-22T10:50:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 61.912359] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 61.917630] nix-daemon[866]: reaped child process 1099, status = succeeded client # [ 61.954789] nix[936]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker2 # [ 61.929924] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # [ 62.153485] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: e7f3e5ee91bc4398ac7b3436435c8b6a client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 1.13 seconds) worker1 # [ 62.387701] nix-grpc-daemon[774]: ts=2026-09-22T10:50:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34278 client # [ 62.432489] nix[952]: don't know how to build these paths: client # [ 62.433432] nix[952]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 62.417895] nix-grpc-daemon[780]: ts=2026-09-22T10:50:12Z level=debug event=want client=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 client # [ 62.453732] nix[952]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker2 # [ 62.425477] nix-grpc-daemon[780]: ts=2026-09-22T10:50:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 65.582801] systemd[1]: share1.service: Sent signal SIGINT to main process 936 (nix) on client request. client # [ 65.591310] nix[936]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.10 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 65.607859] nix[936]: error: interrupted by the user client # [ 65.669968] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 65.674680] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 65.680454] systemd[1]: share1.service: Consumed 181ms CPU time over 4.642s wall clock time, 41.3M memory peak, 6.3K incoming IP traffic, 8.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 0.09 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 66.180653] nix-daemon[866]: reaped child process 1113, status = succeeded worker2 # [ 66.364827] nix-daemon[866]: accepted connection from pid 1149, user nix-grpc-daemon (trusted) worker2 # [ 66.381880] nix-daemon[866]: reaped child process 1152, status = succeeded lb # [ 66.395284] niks3-server[991]: 2026/09/22 10:50:15 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 66.408395] nix-grpc-daemon[880]: time=2026-09-22T10:50:16.008Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 66.409876] nix-grpc-daemon[880]: time=2026-09-22T10:50:16.008Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 66.436127] niks3-server[991]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 66.448536] niks3-server[991]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 66.456282] nix-grpc-daemon[880]: time=2026-09-22T10:50:16.059Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.460565] niks3-server[991]: 2026/09/22 10:50:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 66.463985] niks3-server[991]: 2026/09/22 10:50:16 INFO Signed narinfos id=12 count=1 lb # [ 66.473289] niks3-server[991]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls worker2 # [ 66.473729] nix-grpc-daemon[880]: time=2026-09-22T10:50:16.076Z level=INFO msg="Upload complete. (331ms)" lb # [ 66.478466] niks3-server[991]: 2026/09/22 10:50:16 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 66.482574] niks3-server[991]: 2026/09/22 10:50:16 INFO Completed upload id=12 lb # [ 66.484484] niks3-server[991]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 66.485027] nix-grpc-daemon[780]: ts=2026-09-22T10:50:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=6 outputs=1 worker2 # [ 66.496260] nix-grpc-daemon[780]: ts=2026-09-22T10:50:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=6 outputs=1 worker2 # [ 66.502947] nix-daemon[866]: reaped child process 1111, status = succeeded client # [ 66.597520] systemd[1]: share2.service: Deactivated successfully. client # [ 66.601789] systemd[1]: share2.service: Consumed 174ms CPU time over 4.448s wall clock time, 31.2M memory peak, 5.9K incoming IP traffic, 5.8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.14 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds) (finished: subtest: the first client leaving does not fail the second, in 6.21 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 67.377709] systemd[1]: Started Nix Daemon. client # [ 67.569533] nix-daemon[999]: accepted connection from pid 996, user root (trusted) worker1 # [ 67.747086] nix-grpc-daemon[774]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34278 worker2 # [ 67.761428] nix-grpc-daemon[780]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 67.775381] nix-grpc-daemon[780]: ts=2026-09-22T10:50:17Z level=debug event=want client=8 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker2 # [ 67.781358] nix-grpc-daemon[780]: ts=2026-09-22T10:50:17Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 67.812298] nix-grpc-daemon[774]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 67.827868] nix-grpc-daemon[774]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 67.843082] nix-grpc-daemon[774]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 67.858106] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 68.063637] nix-daemon[839]: accepted connection from pid 958, user nix-grpc-daemon (trusted) worker1 # [ 68.083337] nix-daemon[839]: reaped child process 961, status = succeeded worker1 # [ 68.253315] nix-daemon[839]: accepted connection from pid 963, user nix-grpc-daemon (trusted) worker1 # [ 68.271755] nix-daemon[839]: reaped child process 966, status = succeeded lb # [ 68.292910] niks3-server[991]: 2026/09/22 10:50:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 68.294306] nix-grpc-daemon[867]: time=2026-09-22T10:50:17.896Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 68.295782] nix-grpc-daemon[867]: time=2026-09-22T10:50:17.896Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 68.322021] niks3-server[991]: 2026/09/22 10:50:17 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 68.336592] niks3-server[991]: 2026/09/22 10:50:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign worker1 # [ 68.335535] nix-grpc-daemon[867]: time=2026-09-22T10:50:17.938Z level=INFO msg="Uploading 1 narinfos" lb # [ 68.346635] niks3-server[991]: 2026/09/22 10:50:17 INFO Signed narinfos id=13 count=1 lb # [ 68.352754] niks3-server[991]: 2026/09/22 10:50:17 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 68.364178] niks3-server[991]: 2026/09/22 10:50:17 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete worker1 # [ 68.363051] nix-grpc-daemon[867]: time=2026-09-22T10:50:17.965Z level=INFO msg="Upload complete. (488ms)" lb # [ 68.371152] niks3-server[991]: 2026/09/22 10:50:17 INFO Completed upload id=13 lb # [ 68.374838] niks3-server[991]: 2026/09/22 10:50:17 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 68.372319] nix-grpc-daemon[774]: ts=2026-09-22T10:50:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 68.383787] nix-grpc-daemon[774]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 68.393890] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 68.397057] nix-daemon[839]: reaped child process 956, status = succeeded worker1 # [ 68.410713] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 69.638801] nix-daemon[839]: reaped child process 970, status = succeeded worker1 # [ 69.801240] nix-daemon[839]: accepted connection from pid 986, user nix-grpc-daemon (trusted) worker1 # [ 69.817889] nix-daemon[839]: reaped child process 989, status = succeeded lb # [ 69.838311] niks3-server[991]: 2026/09/22 10:50:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 69.840522] nix-grpc-daemon[867]: time=2026-09-22T10:50:19.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 69.841995] nix-grpc-daemon[867]: time=2026-09-22T10:50:19.442Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 69.874942] niks3-server[991]: 2026/09/22 10:50:19 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 69.886641] niks3-server[991]: 2026/09/22 10:50:19 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 69.897293] niks3-server[991]: 2026/09/22 10:50:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker1 # [ 69.893787] nix-grpc-daemon[867]: time=2026-09-22T10:50:19.496Z level=INFO msg="Uploading 1 narinfos" lb # [ 69.904660] niks3-server[991]: 2026/09/22 10:50:19 INFO Signed narinfos id=14 count=1 lb # [ 69.906765] niks3-server[991]: 2026/09/22 10:50:19 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 69.916074] niks3-server[991]: 2026/09/22 10:50:19 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 69.922161] niks3-server[991]: 2026/09/22 10:50:19 INFO Completed upload id=14 worker1 # [ 69.914114] nix-grpc-daemon[867]: time=2026-09-22T10:50:19.516Z level=INFO msg="Upload complete. (289ms)" lb # [ 69.924552] niks3-server[991]: 2026/09/22 10:50:19 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 69.924912] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=7 outputs=1 worker1 # [ 69.940390] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 69.944509] nix-daemon[839]: reaped child process 968, status = succeeded worker1 # [ 69.964125] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 69.970765] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 70.290670] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 70.298034] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 70.304606] nix-grpc-daemon[867]: time=2026-09-22T10:50:19.906Z level=INFO msg="All 1 paths already cached" worker2 # [ 70.328524] nix-grpc-daemon[780]: ts=2026-09-22T10:50:19Z level=debug event=want client=9 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 70.323303] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker2 # [ 70.329014] nix-grpc-daemon[780]: ts=2026-09-22T10:50:19Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 70.339004] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 70.353021] nix-grpc-daemon[867]: time=2026-09-22T10:50:19.955Z level=INFO msg="All 1 paths already cached" worker1 # [ 70.362354] nix-grpc-daemon[774]: ts=2026-09-22T10:50:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 70.376133] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 70.572634] nix-daemon[839]: accepted connection from pid 993, user nix-grpc-daemon (trusted) worker1 # [ 70.589925] nix-daemon[839]: reaped child process 996, status = succeeded worker1 # [ 70.762973] nix-daemon[839]: accepted connection from pid 998, user nix-grpc-daemon (trusted) worker1 # [ 70.780209] nix-daemon[839]: reaped child process 1001, status = succeeded lb # [ 70.799425] niks3-server[991]: 2026/09/22 10:50:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 70.801956] nix-grpc-daemon[867]: time=2026-09-22T10:50:20.404Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 70.803437] nix-grpc-daemon[867]: time=2026-09-22T10:50:20.404Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 70.827124] niks3-server[991]: 2026/09/22 10:50:20 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst lb # [ 70.843615] niks3-server[991]: 2026/09/22 10:50:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker1 # [ 70.842651] nix-grpc-daemon[867]: time=2026-09-22T10:50:20.444Z level=INFO msg="Uploading 1 narinfos" lb # [ 70.857550] niks3-server[991]: 2026/09/22 10:50:20 INFO Signed narinfos id=15 count=1 worker1 # [ 70.857796] nix-grpc-daemon[867]: time=2026-09-22T10:50:20.460Z level=INFO msg="Upload complete. (464ms)" lb # [ 70.867299] niks3-server[991]: 2026/09/22 10:50:20 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 70.872147] niks3-server[991]: 2026/09/22 10:50:20 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete worker1 # [ 70.864129] nix-grpc-daemon[774]: ts=2026-09-22T10:50:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 70.876576] niks3-server[991]: 2026/09/22 10:50:20 INFO Completed upload id=15 lb # [ 70.878519] niks3-server[991]: 2026/09/22 10:50:20 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 70.876370] nix-grpc-daemon[774]: ts=2026-09-22T10:50:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 70.879620] nix-daemon[839]: reaped child process 991, status = succeeded worker1 # [ 70.886669] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 70.899233] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) lb # [ 71.583319] envoy[927]: conn peer=[2001:db8:1::1]:51364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2977 tx=814 ms=4320 lb # [ 71.588542] envoy[927]: conn peer=[2001:db8:1::1]:51374 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=4144 lb # [ 71.593784] envoy[927]: conn peer=[2001:db8:1::1]:51380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3413 tx=1717 ms=2282 lb # [ 71.599443] envoy[927]: conn peer=[2001:db8:1::1]:51394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=74 worker1 # [ 71.944181] nix-daemon[839]: reaped child process 1005, status = succeeded worker1 # [ 72.112115] nix-daemon[839]: accepted connection from pid 1021, user nix-grpc-daemon (trusted) worker1 # [ 72.128526] nix-daemon[839]: reaped child process 1024, status = succeeded lb # [ 72.153928] niks3-server[991]: 2026/09/22 10:50:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 72.155417] nix-grpc-daemon[867]: time=2026-09-22T10:50:21.756Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 72.157366] nix-grpc-daemon[867]: time=2026-09-22T10:50:21.756Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 72.183760] niks3-server[991]: 2026/09/22 10:50:21 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 72.199420] niks3-server[991]: 2026/09/22 10:50:21 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 72.208861] niks3-server[991]: 2026/09/22 10:50:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign worker1 # [ 72.207617] nix-grpc-daemon[867]: time=2026-09-22T10:50:21.810Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.218686] niks3-server[991]: 2026/09/22 10:50:21 INFO Signed narinfos id=16 count=1 lb # [ 72.224254] niks3-server[991]: 2026/09/22 10:50:21 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 72.233655] niks3-server[991]: 2026/09/22 10:50:21 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 72.243191] niks3-server[991]: 2026/09/22 10:50:21 INFO Completed upload id=16 worker1 # [ 72.234026] nix-grpc-daemon[867]: time=2026-09-22T10:50:21.836Z level=INFO msg="Upload complete. (324ms)" lb # [ 72.246334] niks3-server[991]: 2026/09/22 10:50:21 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 72.245259] nix-grpc-daemon[774]: ts=2026-09-22T10:50:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=8 outputs=1 worker1 # [ 72.265292] nix-grpc-daemon[774]: ts=2026-09-22T10:50:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 72.273671] nix-daemon[839]: reaped child process 1003, status = succeeded worker1 # [ 72.281644] nix-grpc-daemon[774]: ts=2026-09-22T10:50:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 72.287412] nix-grpc-daemon[774]: ts=2026-09-22T10:50:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 72.626013] nix-grpc-daemon[774]: ts=2026-09-22T10:50:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34278 worker2 # [ 72.640804] nix-grpc-daemon[780]: ts=2026-09-22T10:50:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 72.677849] nix-grpc-daemon[780]: ts=2026-09-22T10:50:22Z level=debug event=want client=10 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker2 # [ 72.684499] nix-grpc-daemon[780]: ts=2026-09-22T10:50:22Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 72.679460] nix-grpc-daemon[774]: ts=2026-09-22T10:50:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 72.695570] nix-grpc-daemon[774]: ts=2026-09-22T10:50:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 72.713165] nix-grpc-daemon[867]: time=2026-09-22T10:50:22.315Z level=INFO msg="All 2 paths already cached" worker1 # [ 72.723276] nix-grpc-daemon[774]: ts=2026-09-22T10:50:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 72.737777] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 72.952679] nix-daemon[839]: accepted connection from pid 1028, user nix-grpc-daemon (trusted) worker1 # [ 72.974157] nix-daemon[839]: reaped child process 1031, status = succeeded worker1 # [ 73.164263] nix-daemon[839]: accepted connection from pid 1033, user nix-grpc-daemon (trusted) worker1 # [ 73.183654] nix-daemon[839]: reaped child process 1036, status = succeeded lb # [ 73.205603] niks3-server[991]: 2026/09/22 10:50:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 73.248591] nix-grpc-daemon[867]: time=2026-09-22T10:50:22.850Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 73.250461] nix-grpc-daemon[867]: time=2026-09-22T10:50:22.850Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 73.281665] niks3-server[991]: 2026/09/22 10:50:22 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 73.299588] niks3-server[991]: 2026/09/22 10:50:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker1 # [ 73.298906] nix-grpc-daemon[867]: time=2026-09-22T10:50:22.901Z level=INFO msg="Uploading 1 narinfos" lb # [ 73.312141] niks3-server[991]: 2026/09/22 10:50:22 INFO Signed narinfos id=17 count=1 lb # [ 73.322233] niks3-server[991]: 2026/09/22 10:50:22 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 73.339386] niks3-server[991]: 2026/09/22 10:50:22 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 73.349654] niks3-server[991]: 2026/09/22 10:50:22 INFO Completed upload id=17 worker1 # [ 73.340571] nix-grpc-daemon[867]: time=2026-09-22T10:50:22.943Z level=INFO msg="Upload complete. (584ms)" lb # [ 73.353108] niks3-server[991]: 2026/09/22 10:50:22 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 73.353449] nix-grpc-daemon[774]: ts=2026-09-22T10:50:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 73.368724] nix-grpc-daemon[774]: ts=2026-09-22T10:50:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 73.388254] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 73.400254] nix-daemon[839]: reaped child process 1026, status = succeeded worker1 # [ 73.416552] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 74.689869] nix-daemon[839]: reaped child process 1040, status = succeeded worker1 # [ 74.906896] nix-daemon[839]: accepted connection from pid 1056, user nix-grpc-daemon (trusted) worker1 # [ 74.930001] nix-daemon[839]: reaped child process 1059, status = succeeded lb # [ 74.955100] niks3-server[991]: 2026/09/22 10:50:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 74.958544] nix-grpc-daemon[867]: time=2026-09-22T10:50:24.561Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 74.963624] nix-grpc-daemon[867]: time=2026-09-22T10:50:24.563Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 75.021074] niks3-server[991]: 2026/09/22 10:50:24 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 75.031726] niks3-server[991]: 2026/09/22 10:50:24 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 75.041116] niks3-server[991]: 2026/09/22 10:50:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 75.038912] nix-grpc-daemon[867]: time=2026-09-22T10:50:24.640Z level=INFO msg="Uploading 1 narinfos" lb # [ 75.051398] niks3-server[991]: 2026/09/22 10:50:24 INFO Signed narinfos id=18 count=1 lb # [ 75.060793] niks3-server[991]: 2026/09/22 10:50:24 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls lb # [ 75.072096] niks3-server[991]: 2026/09/22 10:50:24 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 75.083532] niks3-server[991]: 2026/09/22 10:50:24 INFO Completed upload id=18 worker1 # [ 75.075542] nix-grpc-daemon[867]: time=2026-09-22T10:50:24.678Z level=INFO msg="Upload complete. (441ms)" lb # [ 75.086744] niks3-server[991]: 2026/09/22 10:50:24 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 75.089761] nix-grpc-daemon[774]: ts=2026-09-22T10:50:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=9 outputs=1 worker1 # [ 75.115262] nix-grpc-daemon[774]: ts=2026-09-22T10:50:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 75.120685] nix-daemon[839]: reaped child process 1038, status = succeeded worker2 # [ 75.152721] nix-grpc-daemon[780]: ts=2026-09-22T10:50:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 75.180699] nix-grpc-daemon[780]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 75.206395] nix-grpc-daemon[780]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 75.228675] nix-grpc-daemon[780]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker2 # [ 75.253394] nix-grpc-daemon[780]: ts=2026-09-22T10:50:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=1 nar_bytes_out=232 client # [ 75.436287] nix-daemon[999]: reaped child process 1001, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.28 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.42 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.50 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.22 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 76.825781] nix-daemon[999]: accepted connection from pid 1082, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker1 # [ 77.044537] nix-grpc-daemon[774]: ts=2026-09-22T10:50:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:34278 client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 77.058915] nix-grpc-daemon[780]: ts=2026-09-22T10:50:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 77.067966] nix-grpc-daemon[774]: ts=2026-09-22T10:50:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 77.083738] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 77.289983] nix-daemon[839]: accepted connection from pid 1068, user nix-grpc-daemon (trusted) worker1 # [ 77.308748] nix-daemon[839]: reaped child process 1071, status = succeeded worker1 # [ 77.334827] nix-grpc-daemon[867]: time=2026-09-22T10:50:26.937Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 77.336415] nix-grpc-daemon[867]: time=2026-09-22T10:50:26.937Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 77.331893] niks3-server[991]: 2026/09/22 10:50:26 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 77.361701] niks3-server[991]: 2026/09/22 10:50:26 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 77.376995] niks3-server[991]: 2026/09/22 10:50:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker1 # [ 77.374826] nix-grpc-daemon[867]: time=2026-09-22T10:50:26.977Z level=INFO msg="Uploading 1 narinfos" lb # [ 77.389932] niks3-server[991]: 2026/09/22 10:50:26 INFO Signed narinfos id=19 count=1 lb # [ 77.397383] niks3-server[991]: 2026/09/22 10:50:26 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 77.408668] niks3-server[991]: 2026/09/22 10:50:26 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 77.413753] niks3-server[991]: 2026/09/22 10:50:26 INFO Completed upload id=19 lb # [ 77.416881] niks3-server[991]: 2026/09/22 10:50:26 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 77.405340] nix-grpc-daemon[867]: time=2026-09-22T10:50:27.008Z level=INFO msg="Upload complete. (306ms)" worker1 # [ 77.416537] nix-grpc-daemon[774]: ts=2026-09-22T10:50:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 77.429938] nix-daemon[839]: reaped child process 1066, status = succeeded worker2 # [ 77.439929] nix-grpc-daemon[780]: ts=2026-09-22T10:50:27Z level=debug event=want client=11 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker2 # [ 77.447457] nix-grpc-daemon[780]: ts=2026-09-22T10:50:27Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 77.484919] nix-grpc-daemon[780]: ts=2026-09-22T10:50:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 77.500823] nix-grpc-daemon[780]: ts=2026-09-22T10:50:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker2 # [ 77.521575] nix-grpc-daemon[780]: ts=2026-09-22T10:50:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 77.539576] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 77.757519] nix-daemon[866]: accepted connection from pid 1170, user nix-grpc-daemon (trusted) worker2 # [ 77.775422] nix-daemon[866]: reaped child process 1173, status = succeeded worker2 # [ 77.947893] nix-daemon[866]: accepted connection from pid 1175, user nix-grpc-daemon (trusted) worker2 # [ 77.963655] nix-daemon[866]: reaped child process 1178, status = succeeded lb # [ 77.977990] niks3-server[991]: 2026/09/22 10:50:27 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 77.987478] niks3-server[991]: 2026/09/22 10:50:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 77.997460] nix-grpc-daemon[880]: time=2026-09-22T10:50:27.599Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 77.998941] nix-grpc-daemon[880]: time=2026-09-22T10:50:27.599Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker2 # [ 78.000808] nix-grpc-daemon[880]: time=2026-09-22T10:50:27.599Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 78.017129] niks3-server[991]: 2026/09/22 10:50:27 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 78.032974] niks3-server[991]: 2026/09/22 10:50:27 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 78.046717] niks3-server[991]: 2026/09/22 10:50:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 78.052949] niks3-server[991]: 2026/09/22 10:50:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 78.060375] niks3-server[991]: 2026/09/22 10:50:27 INFO Signed narinfos id=20 count=1 worker2 # [ 78.059660] nix-grpc-daemon[880]: time=2026-09-22T10:50:27.661Z level=INFO msg="Uploading 3 narinfos" lb # [ 78.065216] niks3-server[991]: 2026/09/22 10:50:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign lb # [ 78.071084] niks3-server[991]: 2026/09/22 10:50:27 INFO Signed narinfos id=21 count=2 lb # [ 78.075084] niks3-server[991]: 2026/09/22 10:50:27 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 78.088094] niks3-server[991]: 2026/09/22 10:50:27 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 78.098568] niks3-server[991]: 2026/09/22 10:50:27 INFO Completed upload id=20 lb # [ 78.101106] niks3-server[991]: 2026/09/22 10:50:27 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 78.103851] niks3-server[991]: 2026/09/22 10:50:27 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 78.108662] niks3-server[991]: 2026/09/22 10:50:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo worker2 # [ 78.109025] nix-grpc-daemon[880]: time=2026-09-22T10:50:27.712Z level=INFO msg="Upload complete. (547ms)" lb # [ 78.112098] niks3-server[991]: 2026/09/22 10:50:27 INFO Completed upload id=21 lb # [ 78.113738] niks3-server[991]: 2026/09/22 10:50:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo worker2 # [ 78.121821] nix-grpc-daemon[780]: ts=2026-09-22T10:50:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 78.136496] nix-grpc-daemon[780]: ts=2026-09-22T10:50:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 78.149067] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 78.152725] nix-daemon[866]: reaped child process 1168, status = succeeded worker2 # [ 78.167888] nix-grpc-daemon[780]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... worker2 # [ 78.184427] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 79.011379] nix-daemon[866]: reaped child process 1183, status = succeeded worker2 # [ 79.181240] nix-daemon[866]: accepted connection from pid 1199, user nix-grpc-daemon (trusted) worker2 # [ 79.202534] nix-daemon[866]: reaped child process 1202, status = succeeded lb # [ 79.216856] niks3-server[991]: 2026/09/22 10:50:28 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 79.316257] nix-grpc-daemon[880]: time=2026-09-22T10:50:28.918Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 79.344580] niks3-server[991]: 2026/09/22 10:50:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign worker2 # [ 79.353019] nix-grpc-daemon[880]: time=2026-09-22T10:50:28.955Z level=INFO msg="Uploading 1 narinfos" lb # [ 79.356722] niks3-server[991]: 2026/09/22 10:50:28 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 79.365943] niks3-server[991]: 2026/09/22 10:50:28 INFO Signed narinfos id=22 count=1 lb # [ 79.373693] niks3-server[991]: 2026/09/22 10:50:28 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 79.385171] niks3-server[991]: 2026/09/22 10:50:28 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 79.390934] niks3-server[991]: 2026/09/22 10:50:28 INFO Completed upload id=22 worker2 # [ 79.391524] nix-grpc-daemon[880]: time=2026-09-22T10:50:28.994Z level=INFO msg="Upload complete. (391ms)" lb # [ 79.396381] niks3-server[991]: 2026/09/22 10:50:28 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo client # copying 1 paths... worker2 # [ 79.408941] nix-grpc-daemon[780]: ts=2026-09-22T10:50:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=10 outputs=1 client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 79.427769] nix-grpc-daemon[780]: ts=2026-09-22T10:50:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 79.433711] nix-daemon[866]: reaped child process 1180, status = succeeded worker1 # [ 79.442057] nix-grpc-daemon[774]: ts=2026-09-22T10:50:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 79.458897] nix-grpc-daemon[774]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... worker1 # [ 79.479680] nix-grpc-daemon[774]: ts=2026-09-22T10:50:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_out=120 client # [ 79.646176] nix-daemon[999]: reaped child process 1085, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.05 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 79.863118] nix-daemon[999]: accepted connection from pid 1104, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker2 # [ 80.046107] nix-grpc-daemon[780]: ts=2026-09-22T10:50:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45466 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 80.053954] nix-grpc-daemon[780]: ts=2026-09-22T10:50:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 80.061763] nix-grpc-daemon[880]: time=2026-09-22T10:50:29.662Z level=INFO msg="All 1 paths already cached" worker2 # [ 80.074266] nix-grpc-daemon[780]: ts=2026-09-22T10:50:29Z level=debug event=want client=12 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker2 # [ 80.081784] nix-grpc-daemon[780]: ts=2026-09-22T10:50:29Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 80.076061] nix-grpc-daemon[774]: ts=2026-09-22T10:50:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 80.092607] nix-grpc-daemon[774]: ts=2026-09-22T10:50:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 80.113465] nix-grpc-daemon[774]: ts=2026-09-22T10:50:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 80.127976] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 80.151474] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 1/5) worker1 # [ 80.167258] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 2/5) worker1 # [ 80.182906] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 3/5) worker1 # [ 80.257435] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 264 ms (attempt 4/5) worker1 # [ 80.530205] nix-grpc-daemon[774]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 80.537173] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 80.544725] nix-grpc-daemon[774]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 80.761605] nix-daemon[839]: accepted connection from pid 1081, user nix-grpc-daemon (trusted) worker1 # [ 80.779291] nix-daemon[839]: reaped child process 1084, status = succeeded worker1 # [ 80.951127] nix-daemon[839]: accepted connection from pid 1086, user nix-grpc-daemon (trusted) worker1 # [ 80.970047] nix-daemon[839]: reaped child process 1089, status = succeeded lb # [ 80.990947] niks3-server[991]: 2026/09/22 10:50:30 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 80.995351] nix-grpc-daemon[867]: time=2026-09-22T10:50:30.597Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 80.997330] nix-grpc-daemon[867]: time=2026-09-22T10:50:30.599Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 81.024197] niks3-server[991]: 2026/09/22 10:50:30 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst worker1 # [ 81.030038] nix-grpc-daemon[867]: time=2026-09-22T10:50:30.632Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.040608] niks3-server[991]: 2026/09/22 10:50:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 81.048721] niks3-server[991]: 2026/09/22 10:50:30 INFO Signed narinfos id=23 count=1 lb # [ 81.051051] niks3-server[991]: 2026/09/22 10:50:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 81.060848] niks3-server[991]: 2026/09/22 10:50:30 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 81.065349] niks3-server[991]: 2026/09/22 10:50:30 INFO Completed upload id=23 worker1 # [ 81.056298] nix-grpc-daemon[867]: time=2026-09-22T10:50:30.659Z level=INFO msg="Upload complete. (491ms)" lb # [ 81.067615] niks3-server[991]: 2026/09/22 10:50:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 81.064816] nix-grpc-daemon[774]: ts=2026-09-22T10:50:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_in=568 worker1 # [ 81.080406] nix-grpc-daemon[774]: ts=2026-09-22T10:50:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 81.088638] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) worker1 # [ 81.093923] nix-daemon[839]: reaped child process 1074, status = succeeded worker1 # [ 81.105466] nix-daemon[839]: accepted connection from pid 774, user nix-grpc-daemon (trusted) client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv lb # [ 81.586177] envoy[927]: conn peer=[2001:db8:1::1]:51410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=1955 ms=2054 lb # [ 81.591291] envoy[927]: conn peer=[2001:db8:1::1]:36640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=67 lb # [ 81.596114] envoy[927]: conn peer=[2001:db8:1::1]:36658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=181 lb # [ 81.600666] envoy[927]: conn peer=[2001:db8:1::1]:36646 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4354 tx=2203 ms=2684 lb # [ 81.606173] envoy[927]: conn peer=[2001:db8:1::1]:36674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=98 lb # [ 81.610912] envoy[927]: conn peer=[2001:db8:1::1]:36662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4280 tx=1850 ms=2493 client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 82.133797] nix-daemon[839]: reaped child process 1093, status = succeeded worker1 # [ 82.292512] nix-daemon[839]: accepted connection from pid 1109, user nix-grpc-daemon (trusted) worker1 # [ 82.310728] nix-daemon[839]: reaped child process 1112, status = succeeded lb # [ 82.331695] niks3-server[991]: 2026/09/22 10:50:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 82.334470] nix-grpc-daemon[867]: time=2026-09-22T10:50:31.935Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 82.360817] niks3-server[991]: 2026/09/22 10:50:31 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 82.362132] nix-grpc-daemon[867]: time=2026-09-22T10:50:31.964Z level=INFO msg="Uploading 1 narinfos" lb # [ 82.373620] niks3-server[991]: 2026/09/22 10:50:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 82.384412] niks3-server[991]: 2026/09/22 10:50:31 INFO Signed narinfos id=24 count=1 lb # [ 82.389136] niks3-server[991]: 2026/09/22 10:50:31 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 82.396715] niks3-server[991]: 2026/09/22 10:50:31 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 82.400964] niks3-server[991]: 2026/09/22 10:50:31 INFO Completed upload id=24 worker1 # [ 82.392391] nix-grpc-daemon[867]: time=2026-09-22T10:50:31.995Z level=INFO msg="Upload complete. (274ms)" lb # [ 82.403647] niks3-server[991]: 2026/09/22 10:50:31 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo client # copying 1 paths... worker1 # [ 82.404676] nix-grpc-daemon[774]: ts=2026-09-22T10:50:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=11 outputs=1 worker2 # [ 82.417811] nix-grpc-daemon[780]: ts=2026-09-22T10:50:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45466 worker1 # [ 82.413875] nix-daemon[839]: reaped child process 1091, status = succeeded worker2 # [ 82.431025] nix-grpc-daemon[780]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 82.468117] nix-grpc-daemon[774]: ts=2026-09-22T10:50:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 82.473839] nix-grpc-daemon[774]: ts=2026-09-22T10:50:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_out=120 client # [ 82.595656] nix-daemon[999]: reaped child process 1107, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.95 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 82.806007] nix-daemon[999]: accepted connection from pid 1126, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 82.987590] nix-grpc-daemon[780]: ts=2026-09-22T10:50:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45466 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 82.983647] nix-grpc-daemon[774]: ts=2026-09-22T10:50:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 82.993268] nix-grpc-daemon[867]: time=2026-09-22T10:50:32.596Z level=INFO msg="All 1 paths already cached" worker2 # [ 83.019132] nix-grpc-daemon[780]: ts=2026-09-22T10:50:32Z level=debug event=want client=13 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker2 # [ 83.026925] nix-grpc-daemon[780]: ts=2026-09-22T10:50:32Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 83.036640] nix-grpc-daemon[780]: ts=2026-09-22T10:50:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 83.047383] nix-grpc-daemon[780]: ts=2026-09-22T10:50:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying 1 paths to 'lb:50051'... worker2 # [ 83.063623] nix-grpc-daemon[880]: time=2026-09-22T10:50:32.664Z level=INFO msg="All 1 paths already cached" client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 83.069302] nix-grpc-daemon[780]: ts=2026-09-22T10:50:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 83.085114] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 83.294433] nix-daemon[866]: accepted connection from pid 1211, user nix-grpc-daemon (trusted) worker2 # [ 83.313154] nix-daemon[866]: reaped child process 1214, status = succeeded worker2 # [ 83.489309] nix-daemon[866]: accepted connection from pid 1216, user nix-grpc-daemon (trusted) worker2 # [ 83.506822] nix-daemon[866]: reaped child process 1219, status = succeeded lb # [ 83.519090] niks3-server[991]: 2026/09/22 10:50:33 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 83.530719] nix-grpc-daemon[880]: time=2026-09-22T10:50:33.133Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 83.532396] nix-grpc-daemon[880]: time=2026-09-22T10:50:33.133Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 83.548704] niks3-server[991]: 2026/09/22 10:50:33 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 83.562637] nix-grpc-daemon[880]: time=2026-09-22T10:50:33.165Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.563104] niks3-server[991]: 2026/09/22 10:50:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign lb # [ 83.574998] niks3-server[991]: 2026/09/22 10:50:33 INFO Signed narinfos id=25 count=1 worker2 # [ 83.584601] nix-grpc-daemon[880]: time=2026-09-22T10:50:33.187Z level=INFO msg="Upload complete. (483ms)" lb # [ 83.584784] niks3-server[991]: 2026/09/22 10:50:33 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 83.589420] niks3-server[991]: 2026/09/22 10:50:33 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 83.593813] niks3-server[991]: 2026/09/22 10:50:33 INFO Completed upload id=25 lb # [ 83.595843] niks3-server[991]: 2026/09/22 10:50:33 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 83.590784] nix-grpc-daemon[780]: ts=2026-09-22T10:50:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 83.603387] nix-grpc-daemon[780]: ts=2026-09-22T10:50:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 worker2 # [ 83.611742] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker2 # [ 83.614474] nix-daemon[866]: reaped child process 1209, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker2 # [ 83.626721] nix-daemon[866]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 84.870220] nix-daemon[866]: reaped child process 1223, status = succeeded worker2 # [ 85.053600] nix-daemon[866]: accepted connection from pid 1239, user nix-grpc-daemon (trusted) worker2 # [ 85.070647] nix-daemon[866]: reaped child process 1242, status = succeeded lb # [ 85.082539] niks3-server[991]: 2026/09/22 10:50:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 85.096073] nix-grpc-daemon[880]: time=2026-09-22T10:50:34.695Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 85.114577] niks3-server[991]: 2026/09/22 10:50:34 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker2 # [ 85.122776] nix-grpc-daemon[880]: time=2026-09-22T10:50:34.725Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.126668] niks3-server[991]: 2026/09/22 10:50:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 85.136729] niks3-server[991]: 2026/09/22 10:50:34 INFO Signed narinfos id=26 count=1 lb # [ 85.139363] niks3-server[991]: 2026/09/22 10:50:34 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 85.147571] niks3-server[991]: 2026/09/22 10:50:34 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 85.153823] niks3-server[991]: 2026/09/22 10:50:34 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker2 # [ 85.158415] nix-grpc-daemon[880]: time=2026-09-22T10:50:34.761Z level=INFO msg="Upload complete. (322ms)" lb # [ 85.158488] niks3-server[991]: 2026/09/22 10:50:34 INFO Completed upload id=26 worker2 # [ 85.165812] nix-grpc-daemon[780]: ts=2026-09-22T10:50:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45466 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=12 outputs=1 worker2 # [ 85.175430] nix-daemon[866]: reaped child process 1221, status = succeeded client # copying 1 paths... worker2 # [ 85.214078] nix-grpc-daemon[780]: ts=2026-09-22T10:50:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45466 client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 85.238038] nix-grpc-daemon[774]: ts=2026-09-22T10:50:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 85.251361] nix-grpc-daemon[774]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... worker1 # [ 85.268227] nix-grpc-daemon[774]: ts=2026-09-22T10:50:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34278 duration_s=0 paths=1 nar_bytes_out=120 client # [ 85.439841] nix-daemon[999]: reaped child process 1129, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.85 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 9.86 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.45 seconds) client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds) worker2 # [ 86.832647] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 87.065346] nix-grpc-daemon[780]: ts=2026-09-22T10:50:36Z level=info event=signal_exit lb # [ 87.338714] niks3-server[991]: 2026/09/22 10:50:36 INFO lead: released remote=192.168.1.4:35718 worker2 # [ 88.337320] nix-grpc-daemon[780]: ts=2026-09-22T10:50:37Z level=info event=scheduler_restarting worker1 # [ 88.330413] nix-grpc-daemon[774]: ts=2026-09-22T10:50:37Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 88.350310] nix-grpc-daemon[780]: ts=2026-09-22T10:50:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45454 worker2 # [ 88.356202] nix-grpc-daemon[780]: ts=2026-09-22T10:50:37Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 88.360861] nix-grpc-daemon[780]: ts=2026-09-22T10:50:37Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 88.365771] nix-grpc-daemon[780]: ts=2026-09-22T10:50:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45454 worker2 # [ 88.370596] nix-grpc-daemon[780]: ts=2026-09-22T10:50:37Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 88.397734] nix-daemon[866]: reaped child process 868, status = succeeded worker1 # [ 88.530592] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 88.535970] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 88.574063] nix-grpc-daemon[780]: ts=2026-09-22T10:50:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 88.737681] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 88.742842] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 88.943683] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 88.948963] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 89.011991] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 89.015342] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 89.026232] systemd[1]: nix-grpc-daemon.service: Consumed 3.301s CPU time over 1min 7.272s wall clock time, 66.3M memory peak, 91.8K incoming IP traffic, 233.6K outgoing IP traffic. worker2 # [ 89.036520] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 89.038444] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 89.041096] systemd[1]: nix-grpc-daemon.socket: Consumed 579us CPU time over 1min 18.108s wall clock time, 256K memory peak, 92.3K incoming IP traffic, 75.4K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 89.149671] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 89.154800] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 89.357405] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 89.361473] nix-grpc-daemon[774]: ts=2026-09-22T10:50:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 89.404941] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34278 worker1 # [ 89.414398] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:34278 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.31 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds) worker2 # [ 89.562472] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 89.558219] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 89.563338] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 89.633778] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 89.764290] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 89.769441] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 89.954739] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=signal_exit worker1 # [ 89.970202] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 89.975183] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 90.175993] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 90.181036] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 90.211896] systemd[1]: Starting scheduler-token.service... worker2 # [ 90.317652] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 90.324222] systemd[1]: Finished scheduler-token.service. worker2 # [ 90.325843] systemd[1]: scheduler-token.service: Consumed 51ms CPU time over 110ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 90.333885] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 90.383559] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 90.387703] nix-grpc-daemon[774]: ts=2026-09-22T10:50:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 90.523732] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 90.526423] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 90.530596] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 90.539473] nix-grpc-daemon[1266]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 90.553712] nix-grpc-daemon[1266]: E0000 00:00:1790074240.140480 1266 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 90.557644] niks3-server[991]: 2026/09/22 10:50:40 INFO lead: acquired remote=192.168.1.4:60416 worker2 # [ 90.562529] nix-grpc-daemon[1266]: E0000 00:00:1790074240.142532 1266 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 90.591861] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 90.584521] nix-grpc-daemon[774]: ts=2026-09-22T10:50:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 90.595759] nix-grpc-daemon[1266]: E0000 00:00:1790074240.159815 1308 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 90.598586] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=scheduler_take_over worker2 # [ 90.599769] nix-grpc-daemon[1266]: E0000 00:00:1790074240.180212 1308 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 90.608240] nix-grpc-daemon[1266]: E0000 00:00:1790074240.180362 1308 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 90.611010] nix-grpc-daemon[1266]: E0000 00:00:1790074240.194948 1308 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 90.613985] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 90.615130] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53888 worker2 # [ 90.618613] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 90.620511] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 90.621766] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 90.622979] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53888 worker2 # [ 90.624686] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 91.205724] nix-grpc-daemon[774]: ts=2026-09-22T10:50:40Z level=info event=scheduler_restarting worker2 # [ 91.223249] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53888 worker2 # [ 91.223613] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:40Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 91.248803] nix-grpc-daemon[774]: ts=2026-09-22T10:50:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 91.265007] nix-daemon[839]: reaped child process 841, status = succeeded lb # [ 91.589044] envoy[927]: conn peer=[2001:db8:1::1]:36684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3739 tx=2013 ms=2452 worker1 # [ 92.067540] nix-grpc-daemon[774]: E0000 00:00:1790074241.669027 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true worker1 # [ 92.123565] nix-grpc-daemon[774]: E0000 00:00:1790074241.726278 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true lb # [ 91.594110] envoy[927]: conn peer=[2001:db8:1::1]:49634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=27 lb # [ 91.594242] envoy[927]: conn peer=[2001:db8:1::1]:49642 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3738 tx=1874 ms=2349 lb # [ 91.594340] envoy[927]: conn peer=[2001:db8:1::1]:49656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=96 lb # [ 91.594438] envoy[927]: conn peer=[2001:db8:1::4]:53772 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=54305 tx=3161 ms=66439 lb # [ 91.594533] envoy[927]: conn peer=[2001:db8:1::1]:49658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=27 lb # [ 91.594628] envoy[927]: conn peer=[2001:db8:1::3]:34896 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12291 tx=3268 ms=69233 worker1 # [ 92.906491] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 92.911230] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 92.916632] systemd[1]: nix-grpc-daemon.service: Consumed 2.693s CPU time over 1min 11.336s wall clock time, 55.9M memory peak, 69K incoming IP traffic, 125.9K outgoing IP traffic. worker1 # [ 92.933182] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 92.938300] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 92.940421] systemd[1]: nix-grpc-daemon.socket: Consumed 837us CPU time over 1min 21.517s wall clock time, 256K memory peak, 76.1K incoming IP traffic, 64.9K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 93.311361] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 93.327013] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 93.354949] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5) worker2 # [ 93.414326] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 2/5) worker2 # [ 93.519384] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5) worker2 # [ 93.644261] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 556 ms (attempt 4/5) worker2 # [ 94.201168] nix-grpc-daemon[1266]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 94.206751] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 94.230853] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 94.247823] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 94.276705] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:43Z level=info event=niks3_push_started pid=1326 worker2 # [ 94.491210] nix-grpc-daemon[1326]: time=2026-09-22T10:50:44.094Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 94.668663] nix-daemon[866]: accepted connection from pid 1339, user nix-grpc-daemon (trusted) worker2 # [ 94.685781] nix-daemon[866]: reaped child process 1342, status = succeeded lb # [ 94.701675] niks3-server[991]: 2026/09/22 10:50:44 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 94.710379] niks3-server[991]: 2026/09/22 10:50:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 94.720077] nix-grpc-daemon[1326]: time=2026-09-22T10:50:44.322Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 94.721624] nix-grpc-daemon[1326]: time=2026-09-22T10:50:44.322Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 94.725489] nix-grpc-daemon[1326]: time=2026-09-22T10:50:44.322Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" lb # [ 94.758528] niks3-server[991]: 2026/09/22 10:50:44 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 94.771322] niks3-server[991]: 2026/09/22 10:50:44 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 94.783843] niks3-server[991]: 2026/09/22 10:50:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign worker2 # [ 94.791394] nix-grpc-daemon[1326]: time=2026-09-22T10:50:44.393Z level=INFO msg="Uploading 3 narinfos" lb # [ 94.791879] niks3-server[991]: 2026/09/22 10:50:44 INFO Signed narinfos id=27 count=1 lb # [ 94.798503] niks3-server[991]: 2026/09/22 10:50:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 94.805025] niks3-server[991]: 2026/09/22 10:50:44 INFO Signed narinfos id=28 count=2 lb # [ 94.807032] niks3-server[991]: 2026/09/22 10:50:44 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 94.815048] niks3-server[991]: 2026/09/22 10:50:44 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 94.828059] niks3-server[991]: 2026/09/22 10:50:44 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 94.836992] niks3-server[991]: 2026/09/22 10:50:44 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 94.844223] niks3-server[991]: 2026/09/22 10:50:44 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 94.851295] niks3-server[991]: 2026/09/22 10:50:44 INFO Completed upload id=27 worker2 # [ 94.850982] nix-grpc-daemon[1326]: time=2026-09-22T10:50:44.454Z level=INFO msg="Upload complete. (355ms)" lb # [ 94.854775] niks3-server[991]: 2026/09/22 10:50:44 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 94.858642] niks3-server[991]: 2026/09/22 10:50:44 INFO Completed upload id=28 worker2 # [ 94.865207] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 94.880232] nix-daemon[866]: reaped child process 1324, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.87 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 95.016795] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 95.023543] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 95.220453] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 95.223367] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 95.227511] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 95.236068] nix-grpc-daemon[1136]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 95.237497] nix-grpc-daemon[1136]: E0000 00:00:1790074244.836798 1136 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 95.239755] nix-grpc-daemon[1136]: E0000 00:00:1790074244.838896 1136 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 95.248045] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 95.251412] systemd[1]: Started Nix worker-protocol over gRPC. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 95.256778] nix-grpc-daemon[1136]: E0000 00:00:1790074244.859049 1175 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 95.284185] nix-grpc-daemon[1136]: E0000 00:00:1790074244.876493 1175 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 95.286458] nix-grpc-daemon[1136]: E0000 00:00:1790074244.876667 1175 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 95.294000] nix-grpc-daemon[1136]: E0000 00:00:1790074244.895898 1175 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 95.310672] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 95.321433] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53888 worker2 # [ 95.327102] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.01 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb # [ 96.581060] niks3-server[991]: 2026/09/22 10:50:46 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 96.585047] systemd[1]: Stopping niks3 server... worker1 # [ 96.579898] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:46Z level=info event=scheduler_lock_lost status=200 worker2 # [ 96.589306] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:46Z level=info event=scheduler_lock_lost status=200 lb # [ 96.603318] niks3-server[991]: 2026/09/22 10:50:46 INFO lead: released remote=192.168.1.4:60416 lb # [ 96.607824] systemd[1]: niks3.service: Deactivated successfully. lb # [ 96.609516] systemd[1]: Stopped niks3 server. lb # [ 96.610339] systemd[1]: niks3.service: Consumed 566ms CPU time over 1min 10.997s wall clock time, 22.5M memory peak, 80.9K incoming IP traffic, 113.6K outgoing IP traffic. lb # [ 96.622013] systemd[1]: Starting niks3 server... lb # [ 96.695775] niks3-server[1130]: 2026/09/22 10:50:46 goose: no migrations to run. current version: 20260920000000 lb # [ 96.702122] niks3-server[1130]: 2026/09/22 10:50:46 OK 1_commit_pending_closure.sql (6.86ms) lb # [ 96.705550] niks3-server[1130]: 2026/09/22 10:50:46 OK 2_object_stats_trigger.sql (3.11ms) lb # [ 96.706931] niks3-server[1130]: 2026/09/22 10:50:46 goose: up to current file version: 2 lb # [ 96.708907] niks3-server[1130]: 2026/09/22 10:50:46 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key lb # [ 96.719969] niks3-server[1130]: 2026/09/22 10:50:46 INFO Read proxy enabled — serving cache objects from S3 lb # [ 96.721317] niks3-server[1130]: 2026/09/22 10:50:46 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 96.725685] niks3-server[1130]: 2026/09/22 10:50:46 INFO systemd watchdog enabled interval=15s lb # [ 96.726953] niks3-server[1130]: 2026/09/22 10:50:46 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 96.728345] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 97.339261] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38918 client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker1 # [ 97.345309] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 97.357172] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 97.391674] nix-grpc-daemon[1136]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5) worker1 # [ 97.447850] nix-grpc-daemon[1136]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 2/5) worker1 # [ 97.496381] nix-grpc-daemon[1136]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 315 ms (attempt 3/5) lb # [ 97.601350] niks3-server[1130]: 2026/09/22 10:50:47 INFO lead: acquired remote=192.168.1.4:57422 worker1 # [ 97.819991] nix-grpc-daemon[1136]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 589 ms (attempt 4/5) worker1 # [ 98.410192] nix-grpc-daemon[1136]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 98.416160] nix-grpc-daemon[1136]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 98.474954] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:48Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 98.475254] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:48Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 98.517726] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 98.534040] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38918 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 98.548104] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 98.565876] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 98.783157] nix-daemon[866]: accepted connection from pid 1361, user nix-grpc-daemon (trusted) worker2 # [ 98.799880] nix-daemon[866]: reaped child process 1364, status = succeeded worker2 # [ 98.974725] nix-daemon[866]: accepted connection from pid 1366, user nix-grpc-daemon (trusted) worker2 # [ 98.991156] nix-daemon[866]: reaped child process 1369, status = succeeded lb # [ 99.003701] niks3-server[1130]: 2026/09/22 10:50:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 99.020430] nix-grpc-daemon[1326]: time=2026-09-22T10:50:48.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 99.021933] nix-grpc-daemon[1326]: time=2026-09-22T10:50:48.623Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 99.049211] niks3-server[1130]: 2026/09/22 10:50:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 99.063262] niks3-server[1130]: 2026/09/22 10:50:48 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 99.085330] niks3-server[1130]: 2026/09/22 10:50:48 INFO Signed narinfos id=29 count=1 worker2 # [ 99.088652] nix-grpc-daemon[1326]: time=2026-09-22T10:50:48.691Z level=INFO msg="Uploading 1 narinfos" lb # [ 99.098438] niks3-server[1130]: 2026/09/22 10:50:48 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 99.108636] niks3-server[1130]: 2026/09/22 10:50:48 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 99.113392] niks3-server[1130]: 2026/09/22 10:50:48 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo lb # [ 99.116388] niks3-server[1130]: 2026/09/22 10:50:48 INFO Completed upload id=29 worker2 # [ 99.117167] nix-grpc-daemon[1326]: time=2026-09-22T10:50:48.720Z level=INFO msg="Upload complete. (534ms)" worker2 # [ 99.128310] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 99.144487] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 99.152701] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 99.163536] nix-daemon[866]: reaped child process 1359, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 99.171526] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 100.396335] nix-daemon[866]: reaped child process 1373, status = succeeded worker2 # [ 100.562670] nix-daemon[866]: accepted connection from pid 1389, user nix-grpc-daemon (trusted) worker2 # [ 100.580828] nix-daemon[866]: reaped child process 1392, status = succeeded lb # [ 100.595661] niks3-server[1130]: 2026/09/22 10:50:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.611292] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.212Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 100.612962] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.212Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 100.641330] niks3-server[1130]: 2026/09/22 10:50:50 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 100.652426] niks3-server[1130]: 2026/09/22 10:50:50 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 100.662596] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.264Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.661272] niks3-server[1130]: 2026/09/22 10:50:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 100.666152] niks3-server[1130]: 2026/09/22 10:50:50 INFO Signed narinfos id=30 count=1 lb # [ 100.670158] niks3-server[1130]: 2026/09/22 10:50:50 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 100.676471] niks3-server[1130]: 2026/09/22 10:50:50 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 100.685590] niks3-server[1130]: 2026/09/22 10:50:50 INFO Completed upload id=30 worker2 # [ 100.686272] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.289Z level=INFO msg="Upload complete. (302ms)" lb # [ 100.688531] niks3-server[1130]: 2026/09/22 10:50:50 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 100.704552] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 100.718882] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 100.722521] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 100.726190] nix-daemon[866]: reaped child process 1371, status = succeeded worker2 # [ 100.748741] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 100.763496] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 100.779048] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.382Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 100.789574] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 100.803231] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 101.001450] nix-daemon[866]: accepted connection from pid 1398, user nix-grpc-daemon (trusted) worker2 # [ 101.018102] nix-daemon[866]: reaped child process 1401, status = succeeded worker2 # [ 101.188895] nix-daemon[866]: accepted connection from pid 1403, user nix-grpc-daemon (trusted) worker2 # [ 101.206562] nix-daemon[866]: reaped child process 1406, status = succeeded lb # [ 101.217940] niks3-server[1130]: 2026/09/22 10:50:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 101.232090] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.834Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 101.233564] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.834Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 101.252634] niks3-server[1130]: 2026/09/22 10:50:50 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst worker2 # [ 101.265868] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.868Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.265687] niks3-server[1130]: 2026/09/22 10:50:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign lb # [ 101.271274] niks3-server[1130]: 2026/09/22 10:50:50 INFO Signed narinfos id=31 count=1 lb # [ 101.279662] niks3-server[1130]: 2026/09/22 10:50:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 101.291426] niks3-server[1130]: 2026/09/22 10:50:50 INFO Completed upload id=31 worker2 # [ 101.292045] nix-grpc-daemon[1326]: time=2026-09-22T10:50:50.895Z level=INFO msg="Upload complete. (473ms)" lb # [ 101.294072] niks3-server[1130]: 2026/09/22 10:50:50 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 101.298306] niks3-server[1130]: 2026/09/22 10:50:50 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 101.298976] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 101.311700] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 101.318243] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 101.322963] nix-daemon[866]: reaped child process 1396, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker2 # [ 101.335368] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 101.585227] envoy[927]: conn peer=[2001:db8:1::1]:36956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=304 ms=1567 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 102.170445] nix-daemon[866]: reaped child process 1410, status = succeeded worker2 # [ 102.359022] nix-daemon[866]: accepted connection from pid 1426, user nix-grpc-daemon (trusted) worker2 # [ 102.376201] nix-daemon[866]: reaped child process 1429, status = succeeded lb # [ 102.389742] niks3-server[1130]: 2026/09/22 10:50:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 102.401695] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.004Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 102.403262] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.004Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 102.429794] niks3-server[1130]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst worker2 # [ 102.449700] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.052Z level=INFO msg="Uploading 1 narinfos" lb # [ 102.441192] niks3-server[1130]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 102.452877] niks3-server[1130]: 2026/09/22 10:50:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign lb # [ 102.461417] niks3-server[1130]: 2026/09/22 10:50:52 INFO Signed narinfos id=32 count=1 lb # [ 102.465458] niks3-server[1130]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls worker2 # [ 102.474284] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.077Z level=INFO msg="Upload complete. (325ms)" lb # [ 102.475317] niks3-server[1130]: 2026/09/22 10:50:52 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 102.481255] niks3-server[1130]: 2026/09/22 10:50:52 INFO Completed upload id=32 lb # [ 102.484234] niks3-server[1130]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker2 # [ 102.489885] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 102.509639] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 102.514464] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 102.519473] nix-daemon[866]: reaped child process 1408, status = succeeded worker2 # [ 102.548945] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 102.565023] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 102.579269] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.182Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 102.590341] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 102.604395] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 102.802605] nix-daemon[866]: accepted connection from pid 1435, user nix-grpc-daemon (trusted) worker2 # [ 102.821209] nix-daemon[866]: reaped child process 1438, status = succeeded worker2 # [ 102.993585] nix-daemon[866]: accepted connection from pid 1440, user nix-grpc-daemon (trusted) worker2 # [ 103.009959] nix-daemon[866]: reaped child process 1443, status = succeeded lb # [ 103.020792] niks3-server[1130]: 2026/09/22 10:50:52 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 103.031024] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.633Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 103.032603] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.634Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 103.051448] niks3-server[1130]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst lb # [ 103.061873] niks3-server[1130]: 2026/09/22 10:50:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker2 # [ 103.070625] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.673Z level=INFO msg="Uploading 1 narinfos" lb # [ 103.072709] niks3-server[1130]: 2026/09/22 10:50:52 INFO Signed narinfos id=33 count=1 lb # [ 103.075898] niks3-server[1130]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 103.091761] niks3-server[1130]: 2026/09/22 10:50:52 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete worker2 # [ 103.097236] nix-grpc-daemon[1326]: time=2026-09-22T10:50:52.700Z level=INFO msg="Upload complete. (475ms)" lb # [ 103.097498] niks3-server[1130]: 2026/09/22 10:50:52 INFO Completed upload id=33 lb # [ 103.099800] niks3-server[1130]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 103.104180] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38918 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 103.116615] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 worker2 # [ 103.125926] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 103.128915] nix-daemon[866]: reaped child process 1433, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 103.142623] nix-daemon[866]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 104.190008] nix-daemon[866]: reaped child process 1447, status = succeeded worker2 # [ 104.361058] nix-daemon[866]: accepted connection from pid 1463, user nix-grpc-daemon (trusted) worker2 # [ 104.378180] nix-daemon[866]: reaped child process 1466, status = succeeded lb # [ 104.391232] niks3-server[1130]: 2026/09/22 10:50:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 104.404415] nix-grpc-daemon[1326]: time=2026-09-22T10:50:54.006Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 104.406102] nix-grpc-daemon[1326]: time=2026-09-22T10:50:54.006Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 104.430001] niks3-server[1130]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 104.446369] niks3-server[1130]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 104.454624] niks3-server[1130]: 2026/09/22 10:50:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign worker2 # [ 104.460696] nix-grpc-daemon[1326]: time=2026-09-22T10:50:54.063Z level=INFO msg="Uploading 1 narinfos" lb # [ 104.463116] niks3-server[1130]: 2026/09/22 10:50:54 INFO Signed narinfos id=34 count=1 lb # [ 104.468635] niks3-server[1130]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 104.475127] niks3-server[1130]: 2026/09/22 10:50:54 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 104.481657] niks3-server[1130]: 2026/09/22 10:50:54 INFO Completed upload id=34 worker2 # [ 104.482325] nix-grpc-daemon[1326]: time=2026-09-22T10:50:54.085Z level=INFO msg="Upload complete. (329ms)" lb # [ 104.484285] niks3-server[1130]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 104.497131] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38918 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 104.512264] nix-daemon[866]: reaped child process 1445, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2, in 7.47 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath, in 0.23 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 8.34 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2 # [ 105.093906] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 105.559494] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:55Z level=info event=signal_exit lb # [ 105.813024] niks3-server[1130]: 2026/09/22 10:50:55 INFO lead: released remote=192.168.1.4:57422 worker1 # [ 106.805193] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:56Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 106.817040] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:56Z level=info event=scheduler_restarting worker2 # [ 106.817436] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:56Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 106.817574] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53888 worker2 # [ 106.817686] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:56Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 106.817808] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53888 worker2 # [ 106.817910] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:56Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 106.865547] nix-daemon[866]: reaped child process 1318, status = succeeded worker1 # [ 107.004540] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 107.009783] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 107.033070] nix-grpc-daemon[1266]: ts=2026-09-22T10:50:56Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 107.210622] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 107.215854] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 107.416735] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 107.421929] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 107.585937] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_take_over lb # [ 107.594991] niks3-server[1130]: 2026/09/22 10:50:57 INFO lead: acquired remote=192.168.1.3:49700 worker2 # [ 107.601226] nix-grpc-daemon[1266]: E0000 00:00:1790074257.204145 1311 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker2 # [ 107.607969] nix-grpc-daemon[1266]: E0000 00:00:1790074257.204373 1311 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 107.622720] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 107.627903] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 107.830557] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 107.834653] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 108.031040] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 108.040132] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:44578 worker1 # [ 108.045392] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 108.947637] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 108.951198] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 108.956390] systemd[1]: nix-grpc-daemon.service: Consumed 1.522s CPU time over 18.606s wall clock time, 36.4M memory peak, 45.5K incoming IP traffic, 76.4K outgoing IP traffic. worker2 # [ 108.967798] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 108.973278] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 108.976882] systemd[1]: nix-grpc-daemon.socket: Consumed 729us CPU time over 19.414s wall clock time, 256K memory peak, 39.4K incoming IP traffic, 29.2K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2 worker1 # [ 109.378965] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:32778 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 109.390720] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 109.417795] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 109.424886] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 109.464134] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 109.480403] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32778 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 109.496109] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 109.511403] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 109.538782] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=info event=niks3_push_started pid=1209 worker1 # [ 109.756985] nix-grpc-daemon[1209]: time=2026-09-22T10:50:59.359Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca="" worker1 # [ 109.938879] nix-daemon[839]: accepted connection from pid 1222, user nix-grpc-daemon (trusted) worker1 # [ 109.960293] nix-daemon[839]: reaped child process 1225, status = succeeded worker1 # [ 110.147513] nix-daemon[839]: accepted connection from pid 1227, user nix-grpc-daemon (trusted) worker1 # [ 110.164996] nix-daemon[839]: reaped child process 1230, status = succeeded lb # [ 110.189429] niks3-server[1130]: 2026/09/22 10:50:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 110.194012] nix-grpc-daemon[1209]: time=2026-09-22T10:50:59.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 110.195556] nix-grpc-daemon[1209]: time=2026-09-22T10:50:59.795Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 110.232494] niks3-server[1130]: 2026/09/22 10:50:59 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 110.248561] niks3-server[1130]: 2026/09/22 10:50:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign worker1 # [ 110.250425] nix-grpc-daemon[1209]: time=2026-09-22T10:50:59.852Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.262611] niks3-server[1130]: 2026/09/22 10:50:59 INFO Signed narinfos id=35 count=1 lb # [ 110.273141] niks3-server[1130]: 2026/09/22 10:50:59 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 110.288831] niks3-server[1130]: 2026/09/22 10:50:59 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 110.298448] niks3-server[1130]: 2026/09/22 10:50:59 INFO Completed upload id=35 worker1 # [ 110.289649] nix-grpc-daemon[1209]: time=2026-09-22T10:50:59.892Z level=INFO msg="Upload complete. (528ms)" lb # [ 110.301691] niks3-server[1130]: 2026/09/22 10:50:59 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker1 # [ 110.300489] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 110.320913] nix-grpc-daemon[1136]: ts=2026-09-22T10:50:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 110.329760] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 110.335383] nix-daemon[839]: reaped child process 1207, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker1 # [ 110.351103] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 111.587515] envoy[927]: conn peer=[2001:db8:1::1]:36970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9710 tx=4139 ms=7166 lb # [ 111.593042] envoy[927]: conn peer=[2001:db8:1::4]:37642 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1691 tx=425 ms=16418 client # farm-a> LOG-farm-a-on-standby worker1 # [ 111.806658] nix-daemon[839]: reaped child process 1234, status = succeeded worker1 # [ 111.984960] nix-daemon[839]: accepted connection from pid 1250, user nix-grpc-daemon (trusted) worker1 # [ 112.001568] nix-daemon[839]: reaped child process 1253, status = succeeded lb # [ 112.023788] niks3-server[1130]: 2026/09/22 10:51:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 112.027841] nix-grpc-daemon[1209]: time=2026-09-22T10:51:01.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 112.030066] nix-grpc-daemon[1209]: time=2026-09-22T10:51:01.627Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 112.064371] niks3-server[1130]: 2026/09/22 10:51:01 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 112.080735] niks3-server[1130]: 2026/09/22 10:51:01 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 112.092065] niks3-server[1130]: 2026/09/22 10:51:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 112.095698] niks3-server[1130]: 2026/09/22 10:51:01 INFO Signed narinfos id=36 count=1 worker1 # [ 112.091730] nix-grpc-daemon[1209]: time=2026-09-22T10:51:01.693Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.107642] niks3-server[1130]: 2026/09/22 10:51:01 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 112.118297] niks3-server[1130]: 2026/09/22 10:51:01 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 112.123138] niks3-server[1130]: 2026/09/22 10:51:01 INFO Completed upload id=36 worker1 # [ 112.114381] nix-grpc-daemon[1209]: time=2026-09-22T10:51:01.717Z level=INFO msg="Upload complete. (344ms)" lb # [ 112.126420] niks3-server[1130]: 2026/09/22 10:51:01 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker1 # [ 112.137441] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 112.152278] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:01Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 112.155692] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:01Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 112.160825] nix-daemon[839]: reaped child process 1232, status = succeeded worker1 # [ 112.179544] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 112.190827] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32778 client # copying 1 paths to 'lb:50051'... worker1 # [ 112.204004] nix-grpc-daemon[1209]: time=2026-09-22T10:51:01.806Z level=INFO msg="All 1 paths already cached" client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 112.209860] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 112.223190] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 112.421882] nix-daemon[839]: accepted connection from pid 1259, user nix-grpc-daemon (trusted) worker1 # [ 112.439999] nix-daemon[839]: reaped child process 1262, status = succeeded worker1 # [ 112.615455] nix-daemon[839]: accepted connection from pid 1264, user nix-grpc-daemon (trusted) worker1 # [ 112.630956] nix-daemon[839]: reaped child process 1267, status = succeeded lb # [ 112.650980] niks3-server[1130]: 2026/09/22 10:51:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 112.652676] nix-grpc-daemon[1209]: time=2026-09-22T10:51:02.255Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 112.654130] nix-grpc-daemon[1209]: time=2026-09-22T10:51:02.255Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 112.677918] niks3-server[1130]: 2026/09/22 10:51:02 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker1 # [ 112.692670] nix-grpc-daemon[1209]: time=2026-09-22T10:51:02.295Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.694090] niks3-server[1130]: 2026/09/22 10:51:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 112.706353] niks3-server[1130]: 2026/09/22 10:51:02 INFO Signed narinfos id=37 count=1 lb # [ 112.713456] niks3-server[1130]: 2026/09/22 10:51:02 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 112.723343] niks3-server[1130]: 2026/09/22 10:51:02 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete worker1 # [ 112.720634] nix-grpc-daemon[1209]: time=2026-09-22T10:51:02.323Z level=INFO msg="Upload complete. (482ms)" lb # [ 112.729771] niks3-server[1130]: 2026/09/22 10:51:02 INFO Completed upload id=37 lb # [ 112.733360] niks3-server[1130]: 2026/09/22 10:51:02 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker1 # [ 112.728160] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 112.741773] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 112.747822] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 112.753106] nix-daemon[839]: reaped child process 1257, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker1 # [ 112.766885] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 113.585782] nix-daemon[839]: reaped child process 1271, status = succeeded worker1 # [ 113.740003] nix-daemon[839]: accepted connection from pid 1287, user nix-grpc-daemon (trusted) worker1 # [ 113.758232] nix-daemon[839]: reaped child process 1290, status = succeeded lb # [ 113.779545] niks3-server[1130]: 2026/09/22 10:51:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 113.781317] nix-grpc-daemon[1209]: time=2026-09-22T10:51:03.383Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 113.782762] nix-grpc-daemon[1209]: time=2026-09-22T10:51:03.383Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 113.813503] niks3-server[1130]: 2026/09/22 10:51:03 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 113.825390] niks3-server[1130]: 2026/09/22 10:51:03 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 113.835366] niks3-server[1130]: 2026/09/22 10:51:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 113.841427] niks3-server[1130]: 2026/09/22 10:51:03 INFO Signed narinfos id=38 count=1 worker1 # [ 113.832698] nix-grpc-daemon[1209]: time=2026-09-22T10:51:03.435Z level=INFO msg="Uploading 1 narinfos" lb # [ 113.847015] niks3-server[1130]: 2026/09/22 10:51:03 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 113.856255] niks3-server[1130]: 2026/09/22 10:51:03 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 113.860845] niks3-server[1130]: 2026/09/22 10:51:03 INFO Completed upload id=38 worker1 # [ 113.852790] nix-grpc-daemon[1209]: time=2026-09-22T10:51:03.455Z level=INFO msg="Upload complete. (276ms)" lb # [ 113.863168] niks3-server[1130]: 2026/09/22 10:51:03 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker1 # [ 113.867635] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 113.881037] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:03Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 113.884424] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:03Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 113.887804] nix-daemon[839]: reaped child process 1269, status = succeeded worker1 # [ 113.912155] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 113.926338] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 113.940573] nix-grpc-daemon[1209]: time=2026-09-22T10:51:03.543Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 113.950314] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 113.963850] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 114.155578] nix-daemon[839]: accepted connection from pid 1296, user nix-grpc-daemon (trusted) worker1 # [ 114.173967] nix-daemon[839]: reaped child process 1299, status = succeeded worker1 # [ 114.347875] nix-daemon[839]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 114.364535] nix-daemon[839]: reaped child process 1304, status = succeeded lb # [ 114.384357] niks3-server[1130]: 2026/09/22 10:51:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 114.385174] nix-grpc-daemon[1209]: time=2026-09-22T10:51:03.987Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 114.386647] nix-grpc-daemon[1209]: time=2026-09-22T10:51:03.987Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 114.414257] niks3-server[1130]: 2026/09/22 10:51:03 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst lb # [ 114.426320] niks3-server[1130]: 2026/09/22 10:51:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker1 # [ 114.425494] nix-grpc-daemon[1209]: time=2026-09-22T10:51:04.028Z level=INFO msg="Uploading 1 narinfos" lb # [ 114.436119] niks3-server[1130]: 2026/09/22 10:51:04 INFO Signed narinfos id=39 count=1 lb # [ 114.442068] niks3-server[1130]: 2026/09/22 10:51:04 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 114.450539] niks3-server[1130]: 2026/09/22 10:51:04 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 114.456620] niks3-server[1130]: 2026/09/22 10:51:04 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo lb # [ 114.461728] niks3-server[1130]: 2026/09/22 10:51:04 INFO Completed upload id=39 worker1 # [ 114.453610] nix-grpc-daemon[1209]: time=2026-09-22T10:51:04.056Z level=INFO msg="Upload complete. (474ms)" worker1 # [ 114.461665] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 114.474944] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 114.481220] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 114.485898] nix-daemon[839]: reaped child process 1294, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 114.498960] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 115.521451] nix-daemon[839]: reaped child process 1308, status = succeeded worker1 # [ 115.678244] nix-daemon[839]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker1 # [ 115.695733] nix-daemon[839]: reaped child process 1327, status = succeeded lb # [ 115.716765] niks3-server[1130]: 2026/09/22 10:51:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 115.719402] nix-grpc-daemon[1209]: time=2026-09-22T10:51:05.320Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 115.720992] nix-grpc-daemon[1209]: time=2026-09-22T10:51:05.320Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 115.758076] niks3-server[1130]: 2026/09/22 10:51:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 115.775664] niks3-server[1130]: 2026/09/22 10:51:05 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst lb # [ 115.784234] niks3-server[1130]: 2026/09/22 10:51:05 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 115.783249] nix-grpc-daemon[1209]: time=2026-09-22T10:51:05.385Z level=INFO msg="Uploading 1 narinfos" lb # [ 115.791009] niks3-server[1130]: 2026/09/22 10:51:05 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 115.794909] niks3-server[1130]: 2026/09/22 10:51:05 INFO Signed narinfos id=40 count=1 lb # [ 115.801841] niks3-server[1130]: 2026/09/22 10:51:05 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 115.811655] niks3-server[1130]: 2026/09/22 10:51:05 INFO Completed upload id=40 worker1 # [ 115.803579] nix-grpc-daemon[1209]: time=2026-09-22T10:51:05.406Z level=INFO msg="Upload complete. (295ms)" lb # [ 115.820936] niks3-server[1130]: 2026/09/22 10:51:05 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 115.816940] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 115.828879] nix-daemon[839]: reaped child process 1306, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.75 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds) worker2 # [ 116.204813] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 116.210777] systemd[1]: Starting scheduler-token.service... worker2 # [ 116.309324] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 116.316826] systemd[1]: Finished scheduler-token.service. worker2 # [ 116.318337] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 101ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 116.324466] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 116.508209] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:06Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 116.510804] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 116.514930] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 116.522370] nix-grpc-daemon[1487]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 116.523833] nix-grpc-daemon[1487]: E0000 00:00:1790074266.124743 1487 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 116.535401] nix-grpc-daemon[1487]: E0000 00:00:1790074266.130933 1487 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 116.539811] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:06Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 116.573782] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 116.574744] nix-grpc-daemon[1487]: E0000 00:00:1790074266.143482 1527 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 116.577206] nix-grpc-daemon[1487]: E0000 00:00:1790074266.158569 1527 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 116.579442] nix-grpc-daemon[1487]: E0000 00:00:1790074266.158750 1527 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 116.581971] nix-grpc-daemon[1487]: E0000 00:00:1790074266.173936 1527 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 116.591136] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.592162] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 116.597832] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:44578 worker1 # [ 116.603224] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.15 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs client # [ 117.024730] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: a9a3804f980042408e66ba196b2cc556 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds) worker2 # [ 117.258473] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38780 worker2 # [ 117.269810] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) client # [ 117.318687] nix[1252]: don't know how to build these paths: client # [ 117.321631] nix[1252]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 117.297537] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 117.297831] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 117.340325] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 117.355204] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32778 client # [ 117.404110] nix[1252]: copying 1 paths to 'lb:50051'... client # [ 117.407966] nix[1252]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 117.372106] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 117.387927] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 117.586108] nix-daemon[839]: accepted connection from pid 1340, user nix-grpc-daemon (trusted) worker1 # [ 117.604336] nix-daemon[839]: reaped child process 1343, status = succeeded worker1 # [ 117.777405] nix-daemon[839]: accepted connection from pid 1345, user nix-grpc-daemon (trusted) worker1 # [ 117.795563] nix-daemon[839]: reaped child process 1348, status = succeeded lb # [ 117.815741] niks3-server[1130]: 2026/09/22 10:51:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 117.820376] nix-grpc-daemon[1209]: time=2026-09-22T10:51:07.422Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 117.821810] nix-grpc-daemon[1209]: time=2026-09-22T10:51:07.422Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 117.848072] niks3-server[1130]: 2026/09/22 10:51:07 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 117.860906] niks3-server[1130]: 2026/09/22 10:51:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign worker1 # [ 117.858818] nix-grpc-daemon[1209]: time=2026-09-22T10:51:07.461Z level=INFO msg="Uploading 1 narinfos" lb # [ 117.871302] niks3-server[1130]: 2026/09/22 10:51:07 INFO Signed narinfos id=41 count=1 lb # [ 117.876570] niks3-server[1130]: 2026/09/22 10:51:07 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls lb # [ 117.883891] niks3-server[1130]: 2026/09/22 10:51:07 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 117.890348] niks3-server[1130]: 2026/09/22 10:51:07 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo lb # [ 117.894334] niks3-server[1130]: 2026/09/22 10:51:07 INFO Completed upload id=41 worker1 # [ 117.885912] nix-grpc-daemon[1209]: time=2026-09-22T10:51:07.488Z level=INFO msg="Upload complete. (481ms)" worker1 # [ 117.893975] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 117.907415] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 worker1 # [ 117.913965] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) worker1 # [ 117.918413] nix-daemon[839]: reaped child process 1338, status = succeeded worker1 # [ 117.928849] nix-daemon[839]: accepted connection from pid 1136, user nix-grpc-daemon (trusted) client # [ 117.963972] nix[1252]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 119.940094] systemd-journald[267]: Received client request to rotate journal, rotating. worker2 # [ 119.958935] systemd-journald[267]: Vacuuming done, freed 0B of archived journals from /var/log/journal/773f8e80e43b44ba9aa84949c58ca24f. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 120.010617] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 worker1 # [ 120.255396] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:09Z level=info event=draining lb # [ 121.586635] envoy[927]: conn peer=[2001:db8:1::1]:50644 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=4094 ms=6449 client: (finished: must succeed: sleep 5.0, in 5.06 seconds) worker1 # [ 125.359782] nix-daemon[839]: reaped child process 1352, status = succeeded worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 125.517957] nix-daemon[839]: accepted connection from pid 1393, user nix-grpc-daemon (trusted) worker1 # [ 125.535781] nix-daemon[839]: reaped child process 1396, status = succeeded lb # [ 125.560980] niks3-server[1130]: 2026/09/22 10:51:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 125.564379] nix-grpc-daemon[1209]: time=2026-09-22T10:51:15.165Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 125.565825] nix-grpc-daemon[1209]: time=2026-09-22T10:51:15.165Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 125.602951] niks3-server[1130]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 125.621929] niks3-server[1130]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 125.621870] nix-grpc-daemon[1209]: time=2026-09-22T10:51:15.224Z level=INFO msg="Uploading 1 narinfos" lb # [ 125.630124] niks3-server[1130]: 2026/09/22 10:51:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 125.635395] niks3-server[1130]: 2026/09/22 10:51:15 INFO Signed narinfos id=42 count=1 lb # [ 125.641229] niks3-server[1130]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 125.648949] niks3-server[1130]: 2026/09/22 10:51:15 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 125.653570] niks3-server[1130]: 2026/09/22 10:51:15 INFO Completed upload id=42 worker1 # [ 125.644837] nix-grpc-daemon[1209]: time=2026-09-22T10:51:15.247Z level=INFO msg="Upload complete. (298ms)" lb # [ 125.655819] niks3-server[1130]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker1 # [ 125.657083] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32778 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 125.672938] nix-daemon[839]: reaped child process 1350, status = succeeded client # [ 125.733165] systemd[1]: rs.service: Deactivated successfully. client # [ 125.737532] systemd[1]: rs.service: Consumed 179ms CPU time over 8.712s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 126.256831] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:15Z level=info event=signal_exit lb # [ 126.828634] niks3-server[1130]: 2026/09/22 10:51:16 INFO lead: released remote=192.168.1.3:49700 worker2 # [ 127.824384] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:17Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 127.819089] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:17Z level=info event=scheduler_restarting worker1 # [ 127.824790] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:17Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 127.829674] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:44578 worker1 # [ 127.835216] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:17Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 127.839762] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:44578 worker1 # [ 127.845036] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:17Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 127.869109] nix-daemon[839]: reaped child process 1188, status = succeeded worker1 # [ 127.984835] nix-grpc-daemon[1136]: E0000 00:00:1790074277.587433 1181 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 127.992718] nix-grpc-daemon[1136]: E0000 00:00:1790074277.587648 1181 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 128.024744] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.030851] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 128.038459] nix-grpc-daemon[1136]: ts=2026-09-22T10:51:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 128.231644] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.236936] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 128.437753] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.442769] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 128.643741] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.649199] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.28 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } worker1 # [ 128.760929] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 128.764461] systemd[1]: Stopped Nix worker-protocol over gRPC. client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' worker1 # [ 128.766867] systemd[1]: nix-grpc-daemon.service: Consumed 1.772s CPU time over 33.735s wall clock time, 37.2M memory peak, 46.8K incoming IP traffic, 87.9K outgoing IP traffic. worker1 # [ 128.787370] systemd[1]: Starting Nix worker-protocol over gRPC... client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 128.851413] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 128.855554] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 128.981688] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 128.984248] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 128.988283] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 128.996070] nix-grpc-daemon[1403]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 128.997512] nix-grpc-daemon[1403]: E0000 00:00:1790074278.597372 1403 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 129.004361] nix-grpc-daemon[1403]: E0000 00:00:1790074278.603562 1403 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 129.008391] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 129.011514] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 129.019003] niks3-server[1130]: 2026/09/22 10:51:18 INFO lead: acquired remote=192.168.1.3:39504 worker1 # [ 129.014843] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=scheduler_take_over worker1 # [ 129.031723] nix-grpc-daemon[1403]: E0000 00:00:1790074278.619496 1445 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 129.052069] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 129.049949] nix-grpc-daemon[1403]: E0000 00:00:1790074278.634503 1445 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 129.057000] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 129.052808] nix-grpc-daemon[1403]: E0000 00:00:1790074278.637231 1445 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 129.055262] nix-grpc-daemon[1403]: E0000 00:00:1790074278.651923 1445 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 129.062855] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 129.257906] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 129.262902] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 129.264215] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 129.269160] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 129.463933] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 129.468996] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 129.469885] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 129.474987] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 129.670003] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:19Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 129.675106] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 129.675853] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 129.680985] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 129.877625] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:19Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 129.880151] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 129.886863] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 129.891014] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:38074 worker1 # [ 129.896367] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 129.901849] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:38074 worker1 # [ 129.906596] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.12 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid client # [ 130.147483] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: d54ccbca82f8404bb81cca76605be3e1 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds) worker1 # [ 130.365965] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38078 worker1 # [ 130.381721] nix-daemon[839]: accepted connection from pid 1403, user nix-grpc-daemon (trusted) client # [ 130.435237] nix[1288]: don't know how to build these paths: client # [ 130.437294] nix[1288]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker1 # [ 130.410598] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:20Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 130.416569] nix-grpc-daemon[1403]: ts=2026-09-22T10:51:20Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 130.465564] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 worker2 # [ 130.484226] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5) worker2 # [ 130.507617] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5) worker2 # [ 130.548388] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 3/5) worker2 # [ 130.596781] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 4/5) worker2 # [ 130.621332] nix-grpc-daemon[1487]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 130.627372] nix-grpc-daemon[1487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 130.646017] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38780 client # [ 130.688264] nix[1288]: copying 1 paths to 'lb:50051'... worker2 # [ 130.660751] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 client # [ 130.690485] nix[1288]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker2 # [ 130.678027] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker2 # [ 130.704998] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:20Z level=info event=niks3_push_started pid=1607 worker2 # [ 130.917521] nix-grpc-daemon[1607]: time=2026-09-22T10:51:20.519Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 131.091386] nix-daemon[866]: accepted connection from pid 1620, user nix-grpc-daemon (trusted) worker2 # [ 131.107497] nix-daemon[866]: reaped child process 1623, status = succeeded worker2 # [ 131.278427] nix-daemon[866]: accepted connection from pid 1625, user nix-grpc-daemon (trusted) worker2 # [ 131.294118] nix-daemon[866]: reaped child process 1628, status = succeeded lb # [ 131.307615] niks3-server[1130]: 2026/09/22 10:51:20 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.319760] nix-grpc-daemon[1607]: time=2026-09-22T10:51:20.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 131.321463] nix-grpc-daemon[1607]: time=2026-09-22T10:51:20.921Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 131.343594] niks3-server[1130]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst worker2 # [ 131.354947] nix-grpc-daemon[1607]: time=2026-09-22T10:51:20.957Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.353859] niks3-server[1130]: 2026/09/22 10:51:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 131.362797] niks3-server[1130]: 2026/09/22 10:51:20 INFO Signed narinfos id=43 count=1 lb # [ 131.371589] niks3-server[1130]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 131.380187] niks3-server[1130]: 2026/09/22 10:51:20 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 131.391702] niks3-server[1130]: 2026/09/22 10:51:20 INFO Completed upload id=43 worker2 # [ 131.392746] nix-grpc-daemon[1607]: time=2026-09-22T10:51:20.995Z level=INFO msg="Upload complete. (471ms)" lb # [ 131.395089] niks3-server[1130]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker2 # [ 131.408770] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 131.422220] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 worker2 # [ 131.430910] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker2 # [ 131.444607] nix-daemon[866]: reaped child process 1605, status = succeeded worker2 # [ 131.449543] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) client # [ 131.477370] nix[1288]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb # [ 131.591944] envoy[927]: conn peer=[2001:db8:1::1]:39220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1428 ms=8421 lb # [ 131.598075] envoy[927]: conn peer=[2001:db8:1::3]:42722 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2143 tx=780 ms=32718 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker1 # [ 132.977223] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1403 (nix-grpc-daemon) on client request. worker1 # [ 132.994712] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 133.009024] niks3-server[1130]: 2026/09/22 10:51:22 INFO lead: released remote=192.168.1.3:39504 worker2 # [ 133.015995] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:22Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))" client # [ 133.044818] nix[1288]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker1 # [ 133.034144] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker1 # [ 133.048368] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker1 # [ 133.050247] systemd[1]: nix-grpc-daemon.service: Consumed 215ms CPU time over 4.245s wall clock time, 14.3M memory peak, 9.8K incoming IP traffic, 11.5K outgoing IP traffic. worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.13 seconds) client: must succeed: sleep 5.0 worker1 # [ 133.054777] nix-daemon[839]: reaped child process 1462, status = succeeded worker2 # [ 133.216327] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 133.211691] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1. worker1 # [ 133.224536] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 133.436440] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 133.439283] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 133.443301] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 133.450612] nix-grpc-daemon[1486]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 133.460621] nix-grpc-daemon[1486]: E0000 00:00:1790074283.052710 1486 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 133.470628] niks3-server[1130]: 2026/09/22 10:51:23 INFO lead: acquired remote=192.168.1.3:59190 worker1 # [ 133.464180] nix-grpc-daemon[1486]: E0000 00:00:1790074283.056226 1486 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.466733] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 133.468526] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=scheduler_take_over worker1 # [ 133.470237] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 133.476062] nix-grpc-daemon[1486]: E0000 00:00:1790074283.076650 1528 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 133.504052] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 133.505350] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:37062 worker1 # [ 133.506992] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=1 worker1 # [ 133.513857] nix-grpc-daemon[1486]: E0000 00:00:1790074283.096307 1528 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.516826] nix-grpc-daemon[1486]: E0000 00:00:1790074283.096437 1528 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 133.519348] nix-grpc-daemon[1486]: E0000 00:00:1790074283.116056 1528 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 133.526603] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 133.528468] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:37062 worker1 # [ 133.530034] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 client: (finished: must succeed: sleep 5.0, in 5.06 seconds) client: waiting for success: ! systemctl is-active mid worker2 # [ 138.479988] nix-daemon[866]: reaped child process 1636, status = succeeded worker2 # [ 138.640325] nix-daemon[866]: accepted connection from pid 1669, user nix-grpc-daemon (trusted) worker2 # [ 138.657551] nix-daemon[866]: reaped child process 1672, status = succeeded lb # [ 138.669630] niks3-server[1130]: 2026/09/22 10:51:28 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 138.684101] nix-grpc-daemon[1607]: time=2026-09-22T10:51:28.284Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 138.685531] nix-grpc-daemon[1607]: time=2026-09-22T10:51:28.284Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 138.705044] niks3-server[1130]: 2026/09/22 10:51:28 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 138.721062] niks3-server[1130]: 2026/09/22 10:51:28 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 138.735694] niks3-server[1130]: 2026/09/22 10:51:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign worker2 # [ 138.737948] nix-grpc-daemon[1607]: time=2026-09-22T10:51:28.339Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.744870] niks3-server[1130]: 2026/09/22 10:51:28 INFO Signed narinfos id=44 count=1 lb # [ 138.747395] niks3-server[1130]: 2026/09/22 10:51:28 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls worker2 # [ 138.765013] nix-grpc-daemon[1607]: time=2026-09-22T10:51:28.368Z level=INFO msg="Upload complete. (297ms)" lb # [ 138.757856] niks3-server[1130]: 2026/09/22 10:51:28 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 138.768083] niks3-server[1130]: 2026/09/22 10:51:28 INFO Completed upload id=44 worker2 # [ 138.775051] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=7 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker2 # [ 138.790717] nix-daemon[866]: reaped child process 1633, status = succeeded client # [ 138.849166] systemd[1]: mid.service: Deactivated successfully. client # [ 138.853992] systemd[1]: mid.service: Consumed 176ms CPU time over 8.705s wall clock time, 41.2M memory peak, 7.6K incoming IP traffic, 8.9K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.55 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr client # [ 139.670603] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 7537de620a6641698dee0f372f261846 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.05 seconds) worker1 # [ 139.871381] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 139.885333] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) client # [ 139.939387] nix[1324]: don't know how to build these paths: client # [ 139.940760] nix[1324]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 139.911771] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:29Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 139.918908] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:29Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 139.960178] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 139.976119] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5) worker1 # [ 139.995486] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 128 ms (attempt 2/5) worker1 # [ 140.130771] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 210 ms (attempt 3/5) worker1 # [ 140.348362] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 400 ms (attempt 4/5) worker1 # [ 140.749341] nix-grpc-daemon[1486]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 140.754678] nix-grpc-daemon[1486]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 140.773421] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 client # [ 140.824316] nix[1324]: copying 1 paths to 'lb:50051'... worker1 # [ 140.788398] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 client # [ 140.824631] nix[1324]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker1 # [ 140.807707] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 140.845898] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:30Z level=info event=niks3_push_started pid=1563 worker1 # [ 141.062327] nix-grpc-daemon[1563]: time=2026-09-22T10:51:30.664Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca="" worker1 # [ 141.232676] nix-daemon[839]: accepted connection from pid 1577, user nix-grpc-daemon (trusted) worker1 # [ 141.250769] nix-daemon[839]: reaped child process 1580, status = succeeded worker1 # [ 141.421095] nix-daemon[839]: accepted connection from pid 1582, user nix-grpc-daemon (trusted) worker1 # [ 141.438735] nix-daemon[839]: reaped child process 1585, status = succeeded lb # [ 141.460648] niks3-server[1130]: 2026/09/22 10:51:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 141.466810] nix-grpc-daemon[1563]: time=2026-09-22T10:51:31.068Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 141.468596] nix-grpc-daemon[1563]: time=2026-09-22T10:51:31.068Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 141.509735] niks3-server[1130]: 2026/09/22 10:51:31 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker1 # [ 141.513057] nix-grpc-daemon[1563]: time=2026-09-22T10:51:31.115Z level=INFO msg="Uploading 1 narinfos" lb # [ 141.524161] niks3-server[1130]: 2026/09/22 10:51:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign lb # [ 141.533722] niks3-server[1130]: 2026/09/22 10:51:31 INFO Signed narinfos id=45 count=1 worker1 # [ 141.535636] nix-grpc-daemon[1563]: time=2026-09-22T10:51:31.138Z level=INFO msg="Upload complete. (469ms)" lb # [ 141.539750] niks3-server[1130]: 2026/09/22 10:51:31 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 141.549605] niks3-server[1130]: 2026/09/22 10:51:31 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 141.554220] niks3-server[1130]: 2026/09/22 10:51:31 INFO Completed upload id=45 worker1 # [ 141.544928] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 141.556442] niks3-server[1130]: 2026/09/22 10:51:31 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker1 # [ 141.559500] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 141.567890] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 141.570710] nix-daemon[839]: reaped child process 1558, status = succeeded worker1 # [ 141.581402] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) client # [ 141.617484] nix[1324]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... lb # [ 141.591103] envoy[927]: conn peer=[2001:db8:1::3]:43162 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=854 tx=362 ms=3953 lb # [ 141.596682] envoy[927]: conn peer=[2001:db8:1::1]:56316 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3091 tx=1496 ms=8420 client: must succeed: systemctl kill -s INT intr client # [ 143.438167] systemd[1]: intr.service: Sent signal SIGINT to main process 1324 (nix) on client request. worker1 # [ 143.409282] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:33Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds) client # [ 143.447551] nix[1324]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 143.452693] nix[1324]: error: interrupted by the user client # [ 143.491039] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 143.496175] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 143.498336] systemd[1]: intr.service: Consumed 155ms CPU time over 3.824s wall clock time, 41.2M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 143.607930] nix-daemon[839]: reaped child process 1587, status = succeeded worker1 # [ 143.612970] nix-daemon[1589]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 143.619806] nix-daemon[839]: reaped child process 1589, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 5.13 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw client # [ 144.837270] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 86bf105c0b044a9baca404e6eb441f76 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds) client # [ 145.102537] nix[1345]: don't know how to build these paths: client # [ 145.103580] nix[1345]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker2 # [ 145.073375] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38780 worker1 # [ 145.081398] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:34Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 145.087431] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:34Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 145.136655] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 worker2 # [ 145.152965] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38780 client # [ 145.192558] nix[1345]: copying 1 paths to 'lb:50051'... client # [ 145.198193] nix[1345]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker2 # [ 145.170269] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 worker2 # [ 145.185388] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker2 # [ 145.391060] nix-daemon[866]: accepted connection from pid 1704, user nix-grpc-daemon (trusted) worker2 # [ 145.409015] nix-daemon[866]: reaped child process 1707, status = succeeded worker2 # [ 145.581765] nix-daemon[866]: accepted connection from pid 1709, user nix-grpc-daemon (trusted) worker2 # [ 145.597974] nix-daemon[866]: reaped child process 1712, status = succeeded lb # [ 145.609985] niks3-server[1130]: 2026/09/22 10:51:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 145.621342] nix-grpc-daemon[1607]: time=2026-09-22T10:51:35.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 145.622891] nix-grpc-daemon[1607]: time=2026-09-22T10:51:35.223Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 145.638530] niks3-server[1130]: 2026/09/22 10:51:35 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst lb # [ 145.651686] niks3-server[1130]: 2026/09/22 10:51:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign worker2 # [ 145.658988] nix-grpc-daemon[1607]: time=2026-09-22T10:51:35.261Z level=INFO msg="Uploading 1 narinfos" lb # [ 145.661381] niks3-server[1130]: 2026/09/22 10:51:35 INFO Signed narinfos id=46 count=1 lb # [ 145.670122] niks3-server[1130]: 2026/09/22 10:51:35 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 145.682337] niks3-server[1130]: 2026/09/22 10:51:35 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 145.687728] niks3-server[1130]: 2026/09/22 10:51:35 INFO Completed upload id=46 worker2 # [ 145.688425] nix-grpc-daemon[1607]: time=2026-09-22T10:51:35.291Z level=INFO msg="Upload complete. (486ms)" lb # [ 145.691857] niks3-server[1130]: 2026/09/22 10:51:35 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker2 # [ 145.700236] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 145.709632] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 worker2 # [ 145.718777] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker2 # [ 145.721214] nix-daemon[866]: reaped child process 1702, status = succeeded client # [ 145.758390] nix[1345]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker2 # [ 145.734342] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted) worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2 worker2 # Checking switch inhibitors... done worker2 # activating the configuration... worker2 # [ 148.271788] nixos[1748]: switching to system configuration /nix/store/h60cql6xyl0psr1b1smnni8r3597yviy-nixos-system-worker2-test worker2 # [ 148.274323] systemd[1]: Stopped target Remote File Systems. worker2 # [ 148.276937] systemd[1]: Stopped target Local File Systems. worker2 # setting up /etc... worker2 # [ 149.035343] systemd[1]: Reload requested from client PID 1748 ('.switch-to-conf') (unit backdoor.service)... worker2 # [ 149.045252] systemd[1]: Reloading... worker2 # [ 149.912529] systemd[1]: Reloading finished in 869 ms. worker2 # restarting sysinit-reactivation.target worker2 # [ 149.989968] systemd[1]: Starting Re-setup tmpfiles on a system that is already running.... worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service worker2 # [ 150.151086] systemd[1]: Finished Re-setup tmpfiles on a system that is already running.. worker2 # [ 150.164330] systemd[1]: Reached target Reactivate sysinit units. worker2 # [ 150.165192] systemd[1]: Reloading D-Bus System Message Bus... worker2 # [ 150.166242] dbus-broker-launch[494]: Caught SIGHUP, trigger reload. worker2 # [ 150.167094] systemd[1]: Reloading Nix worker-protocol over gRPC... worker2 # [ 150.183222] dbus-broker-launch[494]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 150.185689] dbus-broker-launch[494]: NSS returned no entry for 'systemd-timesync' worker2 # [ 150.187568] dbus-broker-launch[494]: Invalid user-name in /nix/store/602n2r0g7ngz2pj9sxi9qamwpihwymx6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 150.214793] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker2 # [ 150.238356] systemd[1]: Reloaded D-Bus System Message Bus. worker2 # [ 150.246991] systemd[1]: Reached target Local File Systems. worker2 # [ 150.250209] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 150.260427] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 150.272140] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 150.273383] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 150.275431] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 150.284130] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 150.285401] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 150.287219] systemd[1]: Reached target Remote File Systems. worker2 # [ 150.289186] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 150.292428] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 150.540716] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:40Z level=info event=draining worker2 # [ 150.762837] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 150.765592] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 150.768469] systemd[1]: Starting linger-users.service... worker2 # [ 150.818518] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 150.819493] systemd[1]: Finished linger-users.service. worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service worker2 # [ 151.072144] nixos[1748]: finished switching to system configuration /nix/store/h60cql6xyl0psr1b1smnni8r3597yviy-nixos-system-worker2-test worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.63 seconds) worker2: must succeed: pgrep -f 'read -t [9]0 x' worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 151.399079] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 151.407316] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 151.432739] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 151.439494] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 151.484779] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 151.499437] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 client # copying 1 paths to 'lb:50051'... worker1 # [ 151.514882] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker1 # [ 151.539280] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) lb # [ 151.589961] envoy[927]: conn peer=[2001:db8:1::1]:40050 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=826 ms=3569 worker1 # [ 151.745117] nix-daemon[839]: accepted connection from pid 1644, user nix-grpc-daemon (trusted) worker1 # [ 151.761918] nix-daemon[839]: reaped child process 1647, status = succeeded worker1 # [ 151.935373] nix-daemon[839]: accepted connection from pid 1649, user nix-grpc-daemon (trusted) worker1 # [ 151.951571] nix-daemon[839]: reaped child process 1652, status = succeeded lb # [ 151.973841] niks3-server[1130]: 2026/09/22 10:51:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 151.974043] nix-grpc-daemon[1563]: time=2026-09-22T10:51:41.576Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 151.975481] nix-grpc-daemon[1563]: time=2026-09-22T10:51:41.576Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" worker1 # [ 152.006905] nix-grpc-daemon[1563]: time=2026-09-22T10:51:41.609Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.002931] niks3-server[1130]: 2026/09/22 10:51:41 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 152.017667] niks3-server[1130]: 2026/09/22 10:51:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 152.028792] niks3-server[1130]: 2026/09/22 10:51:41 INFO Signed narinfos id=47 count=1 lb # [ 152.033991] niks3-server[1130]: 2026/09/22 10:51:41 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls worker1 # [ 152.036869] nix-grpc-daemon[1563]: time=2026-09-22T10:51:41.639Z level=INFO msg="Upload complete. (480ms)" lb # [ 152.043987] niks3-server[1130]: 2026/09/22 10:51:41 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 152.051676] niks3-server[1130]: 2026/09/22 10:51:41 INFO Completed upload id=47 worker1 # [ 152.044109] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=528 lb # [ 152.055573] niks3-server[1130]: 2026/09/22 10:51:41 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker1 # [ 152.055065] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 152.063747] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 152.066634] nix-daemon[839]: reaped child process 1642, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker1 # [ 152.077867] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker1 # [ 153.303867] nix-daemon[839]: reaped child process 1656, status = succeeded worker1 # [ 153.465156] nix-daemon[839]: accepted connection from pid 1672, user nix-grpc-daemon (trusted) worker1 # [ 153.484146] nix-daemon[839]: reaped child process 1675, status = succeeded lb # [ 153.506138] niks3-server[1130]: 2026/09/22 10:51:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 153.508593] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.109Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 153.510070] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.110Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 153.543589] niks3-server[1130]: 2026/09/22 10:51:43 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 153.557796] niks3-server[1130]: 2026/09/22 10:51:43 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 153.569465] niks3-server[1130]: 2026/09/22 10:51:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign worker1 # [ 153.567163] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.169Z level=INFO msg="Uploading 1 narinfos" lb # [ 153.575619] niks3-server[1130]: 2026/09/22 10:51:43 INFO Signed narinfos id=48 count=1 lb # [ 153.581497] niks3-server[1130]: 2026/09/22 10:51:43 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 153.591660] niks3-server[1130]: 2026/09/22 10:51:43 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 153.598756] niks3-server[1130]: 2026/09/22 10:51:43 INFO Completed upload id=48 worker1 # [ 153.590021] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.192Z level=INFO msg="Upload complete. (300ms)" lb # [ 153.601498] niks3-server[1130]: 2026/09/22 10:51:43 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker1 # [ 153.602413] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=1 worker1 # [ 153.620166] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 153.623769] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 153.628173] nix-daemon[839]: reaped child process 1654, status = succeeded worker1 # [ 153.666199] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 153.681371] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 153.698106] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.300Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 153.708305] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker1 # [ 153.732291] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 153.936403] nix-daemon[839]: accepted connection from pid 1681, user nix-grpc-daemon (trusted) worker1 # [ 153.953776] nix-daemon[839]: reaped child process 1684, status = succeeded worker1 # [ 154.127397] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted) worker1 # [ 154.146349] nix-daemon[839]: reaped child process 1689, status = succeeded lb # [ 154.165870] niks3-server[1130]: 2026/09/22 10:51:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 154.166697] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.769Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 154.168854] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.771Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 154.194906] niks3-server[1130]: 2026/09/22 10:51:43 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst lb # [ 154.210048] niks3-server[1130]: 2026/09/22 10:51:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign worker1 # [ 154.208221] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.811Z level=INFO msg="Uploading 1 narinfos" lb # [ 154.217574] niks3-server[1130]: 2026/09/22 10:51:43 INFO Signed narinfos id=49 count=1 lb # [ 154.224427] niks3-server[1130]: 2026/09/22 10:51:43 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 154.238000] niks3-server[1130]: 2026/09/22 10:51:43 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete worker1 # [ 154.238927] nix-grpc-daemon[1563]: time=2026-09-22T10:51:43.841Z level=INFO msg="Upload complete. (489ms)" lb # [ 154.247195] niks3-server[1130]: 2026/09/22 10:51:43 INFO Completed upload id=49 lb # [ 154.250880] niks3-server[1130]: 2026/09/22 10:51:43 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker1 # [ 154.248282] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 154.259975] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 154.267380] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 154.273047] nix-daemon[839]: reaped child process 1679, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker1 # [ 154.287426] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker1 # [ 155.933435] nix-daemon[839]: reaped child process 1693, status = succeeded worker1 # [ 156.114719] nix-daemon[839]: accepted connection from pid 1709, user nix-grpc-daemon (trusted) worker1 # [ 156.131056] nix-daemon[839]: reaped child process 1712, status = succeeded lb # [ 156.152585] niks3-server[1130]: 2026/09/22 10:51:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 156.156869] nix-grpc-daemon[1563]: time=2026-09-22T10:51:45.758Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 156.158435] nix-grpc-daemon[1563]: time=2026-09-22T10:51:45.758Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 156.188985] niks3-server[1130]: 2026/09/22 10:51:45 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 156.202403] niks3-server[1130]: 2026/09/22 10:51:45 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 156.213122] niks3-server[1130]: 2026/09/22 10:51:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign worker1 # [ 156.209239] nix-grpc-daemon[1563]: time=2026-09-22T10:51:45.811Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.220423] niks3-server[1130]: 2026/09/22 10:51:45 INFO Signed narinfos id=50 count=1 lb # [ 156.225121] niks3-server[1130]: 2026/09/22 10:51:45 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 156.231380] niks3-server[1130]: 2026/09/22 10:51:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 156.241560] niks3-server[1130]: 2026/09/22 10:51:45 INFO Completed upload id=50 lb # [ 156.244782] niks3-server[1130]: 2026/09/22 10:51:45 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 156.233567] nix-grpc-daemon[1563]: time=2026-09-22T10:51:45.836Z level=INFO msg="Upload complete. (335ms)" worker1 # [ 156.253035] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=1 worker1 # [ 156.268129] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:45Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker1 # [ 156.271739] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:45Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 156.276137] nix-daemon[839]: reaped child process 1691, status = succeeded worker1 # [ 156.296285] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 156.311120] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 client # copying 1 paths to 'lb:50051'... worker1 # [ 156.325759] nix-grpc-daemon[1563]: time=2026-09-22T10:51:45.928Z level=INFO msg="All 2 paths already cached" client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker1 # [ 156.332112] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 156.351018] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 156.551199] nix-daemon[839]: accepted connection from pid 1718, user nix-grpc-daemon (trusted) worker1 # [ 156.567653] nix-daemon[839]: reaped child process 1721, status = succeeded worker1 # [ 156.739493] nix-daemon[839]: accepted connection from pid 1723, user nix-grpc-daemon (trusted) worker1 # [ 156.755343] nix-daemon[839]: reaped child process 1726, status = succeeded lb # [ 156.776817] niks3-server[1130]: 2026/09/22 10:51:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 156.777031] nix-grpc-daemon[1563]: time=2026-09-22T10:51:46.379Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 156.778499] nix-grpc-daemon[1563]: time=2026-09-22T10:51:46.379Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 156.800373] niks3-server[1130]: 2026/09/22 10:51:46 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 156.811708] niks3-server[1130]: 2026/09/22 10:51:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign worker1 # [ 156.807978] nix-grpc-daemon[1563]: time=2026-09-22T10:51:46.410Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.818823] niks3-server[1130]: 2026/09/22 10:51:46 INFO Signed narinfos id=51 count=1 lb # [ 156.823467] niks3-server[1130]: 2026/09/22 10:51:46 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls worker1 # [ 156.824217] nix-grpc-daemon[1563]: time=2026-09-22T10:51:46.427Z level=INFO msg="Upload complete. (454ms)" lb # [ 156.831632] niks3-server[1130]: 2026/09/22 10:51:46 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 156.836243] niks3-server[1130]: 2026/09/22 10:51:46 INFO Completed upload id=51 lb # [ 156.838513] niks3-server[1130]: 2026/09/22 10:51:46 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker1 # [ 156.830707] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 156.838297] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 156.843689] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 156.847486] nix-daemon[839]: reaped child process 1716, status = succeeded worker1 # [ 156.856120] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker1 # [ 158.101394] nix-daemon[839]: reaped child process 1730, status = succeeded worker1 # [ 158.285985] nix-daemon[839]: accepted connection from pid 1746, user nix-grpc-daemon (trusted) worker1 # [ 158.302787] nix-daemon[839]: reaped child process 1749, status = succeeded lb # [ 158.323674] niks3-server[1130]: 2026/09/22 10:51:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 158.327256] nix-grpc-daemon[1563]: time=2026-09-22T10:51:47.928Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 158.328827] nix-grpc-daemon[1563]: time=2026-09-22T10:51:47.928Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 158.357889] niks3-server[1130]: 2026/09/22 10:51:47 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 158.372490] niks3-server[1130]: 2026/09/22 10:51:47 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst worker1 # [ 158.376630] nix-grpc-daemon[1563]: time=2026-09-22T10:51:47.979Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.384613] niks3-server[1130]: 2026/09/22 10:51:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 158.393702] niks3-server[1130]: 2026/09/22 10:51:47 INFO Signed narinfos id=52 count=1 lb # [ 158.400419] niks3-server[1130]: 2026/09/22 10:51:47 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 158.407694] niks3-server[1130]: 2026/09/22 10:51:47 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 158.414789] niks3-server[1130]: 2026/09/22 10:51:47 INFO Completed upload id=52 worker1 # [ 158.406697] nix-grpc-daemon[1563]: time=2026-09-22T10:51:48.009Z level=INFO msg="Upload complete. (339ms)" lb # [ 158.417660] niks3-server[1130]: 2026/09/22 10:51:48 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker1 # [ 158.421101] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=1 worker1 # [ 158.436996] nix-daemon[839]: reaped child process 1728, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 7.33 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds) worker2: must succeed: pkill -f 'read -t [9]0 x' worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds) client: waiting for success: ! systemctl is-active sw worker2 # [ 158.994842] nix-daemon[866]: reaped child process 1716, status = succeeded worker2 # [ 159.176696] nix-daemon[866]: accepted connection from pid 1888, user nix-grpc-daemon (trusted) worker2 # [ 159.195339] nix-daemon[866]: reaped child process 1891, status = succeeded lb # [ 159.206490] niks3-server[1130]: 2026/09/22 10:51:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 159.216436] nix-grpc-daemon[1607]: time=2026-09-22T10:51:48.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 159.217896] nix-grpc-daemon[1607]: time=2026-09-22T10:51:48.819Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 159.251526] niks3-server[1130]: 2026/09/22 10:51:48 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker2 # [ 159.261641] nix-grpc-daemon[1607]: time=2026-09-22T10:51:48.864Z level=INFO msg="Uploading 1 narinfos" lb # [ 159.266656] niks3-server[1130]: 2026/09/22 10:51:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 159.275024] niks3-server[1130]: 2026/09/22 10:51:48 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 159.283825] niks3-server[1130]: 2026/09/22 10:51:48 INFO Signed narinfos id=53 count=1 worker2 # [ 159.287053] nix-grpc-daemon[1607]: time=2026-09-22T10:51:48.890Z level=INFO msg="Upload complete. (329ms)" lb # [ 159.287144] niks3-server[1130]: 2026/09/22 10:51:48 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 159.292511] niks3-server[1130]: 2026/09/22 10:51:48 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 159.296230] niks3-server[1130]: 2026/09/22 10:51:48 INFO Completed upload id=53 worker2 # [ 159.296851] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38780 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=1 lb # [ 159.298864] niks3-server[1130]: 2026/09/22 10:51:48 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker2 # [ 159.309118] nix-daemon[866]: reaped child process 1714, status = succeeded client # [ 159.372573] systemd[1]: sw.service: Deactivated successfully. client # [ 159.377923] systemd[1]: sw.service: Consumed 183ms CPU time over 14.539s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 159.543370] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:49Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 160.027237] systemd[1]: Starting scheduler-token.service... worker2 # [ 160.114931] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 160.117875] systemd[1]: Finished scheduler-token.service. worker2 # [ 160.118964] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 90ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker1 # [ 160.774973] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:37062 worker2 # [ 160.784920] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:50Z level=info event=scheduler_restarting worker2 # [ 160.809885] nix-grpc-daemon[1487]: ts=2026-09-22T10:51:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 160.833560] nix-daemon[866]: reaped child process 1546, status = succeeded worker2 # [ 160.998026] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 161.003841] systemd[1]: nix-grpc-daemon.service: Consumed 1.120s CPU time over 44.672s wall clock time, 34.5M memory peak, 27K incoming IP traffic, 46.5K outgoing IP traffic. worker2 # [ 161.022059] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 161.228385] nix-grpc-daemon[1908]: ts=2026-09-22T10:51:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 161.230943] nix-grpc-daemon[1908]: ts=2026-09-22T10:51:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 161.234959] nix-grpc-daemon[1908]: ts=2026-09-22T10:51:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 161.242297] nix-grpc-daemon[1908]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 161.243757] nix-grpc-daemon[1908]: E0000 00:00:1790074310.844656 1908 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 161.259802] nix-grpc-daemon[1908]: E0000 00:00:1790074310.850586 1908 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 161.280173] nix-grpc-daemon[1908]: ts=2026-09-22T10:51:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 161.300746] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 161.301596] nix-grpc-daemon[1908]: E0000 00:00:1790074310.867906 1948 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 161.303897] nix-grpc-daemon[1908]: E0000 00:00:1790074310.885598 1948 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.37 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 161.309415] nix-grpc-daemon[1908]: E0000 00:00:1790074310.885748 1948 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 161.313666] nix-grpc-daemon[1908]: E0000 00:00:1790074310.901991 1948 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 161.316237] nix-grpc-daemon[1908]: ts=2026-09-22T10:51:50Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 160.779707] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:50Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 161.307391] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:50Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:37062 worker1 # [ 161.316850] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.78 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 lb # [ 161.587952] envoy[927]: conn peer=[2001:db8:1::1]:56726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9655 tx=4145 ms=7036 lb # [ 161.588262] envoy[927]: conn peer=[2001:db8:1::1]:46256 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1424 ms=14243 lb # [ 161.588356] envoy[927]: conn peer=[2001:db8:1::4]:58122 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9957 tx=743 ms=44208 worker2 # [ 161.786716] nix-grpc-daemon[1908]: ts=2026-09-22T10:51:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 161.800227] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker1 # [ 161.811578] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 161.854157] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker1 # [ 161.861033] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 161.867628] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker1 # [ 161.873125] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 161.899832] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 161.914532] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 161.928396] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 161.943248] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 162.144266] nix-daemon[839]: accepted connection from pid 1758, user nix-grpc-daemon (trusted) worker1 # [ 162.162967] nix-daemon[839]: reaped child process 1761, status = succeeded worker1 # [ 162.332884] nix-daemon[839]: accepted connection from pid 1763, user nix-grpc-daemon (trusted) worker1 # [ 162.350411] nix-daemon[839]: reaped child process 1766, status = succeeded lb # [ 162.373930] niks3-server[1130]: 2026/09/22 10:51:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 162.375543] nix-grpc-daemon[1563]: time=2026-09-22T10:51:51.977Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 162.377098] nix-grpc-daemon[1563]: time=2026-09-22T10:51:51.977Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 162.399644] niks3-server[1130]: 2026/09/22 10:51:51 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst lb # [ 162.415205] niks3-server[1130]: 2026/09/22 10:51:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 162.413273] nix-grpc-daemon[1563]: time=2026-09-22T10:51:52.015Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.427418] niks3-server[1130]: 2026/09/22 10:51:52 INFO Signed narinfos id=54 count=1 lb # [ 162.437545] niks3-server[1130]: 2026/09/22 10:51:52 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 162.435006] nix-grpc-daemon[1563]: time=2026-09-22T10:51:52.037Z level=INFO msg="Upload complete. (475ms)" lb # [ 162.443953] niks3-server[1130]: 2026/09/22 10:51:52 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 162.448957] niks3-server[1130]: 2026/09/22 10:51:52 INFO Completed upload id=54 worker1 # [ 162.440761] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 162.452117] niks3-server[1130]: 2026/09/22 10:51:52 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 162.454178] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 162.460584] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 162.464892] nix-daemon[839]: reaped child process 1756, status = succeeded worker1 # [ 162.477860] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 163.321576] nix-daemon[839]: reaped child process 1770, status = succeeded worker1 # [ 163.492566] nix-daemon[839]: accepted connection from pid 1786, user nix-grpc-daemon (trusted) worker1 # [ 163.507713] nix-daemon[839]: reaped child process 1789, status = succeeded lb # [ 163.533887] niks3-server[1130]: 2026/09/22 10:51:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 163.535122] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.135Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 163.536847] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.135Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 163.566681] niks3-server[1130]: 2026/09/22 10:51:53 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 163.579944] niks3-server[1130]: 2026/09/22 10:51:53 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker1 # [ 163.580113] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.182Z level=INFO msg="Uploading 1 narinfos" lb # [ 163.593181] niks3-server[1130]: 2026/09/22 10:51:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 163.600535] niks3-server[1130]: 2026/09/22 10:51:53 INFO Signed narinfos id=55 count=1 lb # [ 163.606578] niks3-server[1130]: 2026/09/22 10:51:53 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 163.610138] niks3-server[1130]: 2026/09/22 10:51:53 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 163.614869] niks3-server[1130]: 2026/09/22 10:51:53 INFO Completed upload id=55 worker1 # [ 163.606733] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.209Z level=INFO msg="Upload complete. (319ms)" lb # [ 163.617847] niks3-server[1130]: 2026/09/22 10:51:53 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 163.623388] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=1 worker1 # [ 163.640258] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 163.652771] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 163.657388] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 163.666910] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 163.675324] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 163.680805] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 163.684197] nix-daemon[839]: reaped child process 1768, status = succeeded worker1 # [ 163.868727] nix-daemon[839]: accepted connection from pid 1795, user nix-grpc-daemon (trusted) worker1 # [ 163.885724] nix-daemon[839]: reaped child process 1798, status = succeeded worker1 # [ 164.057454] nix-daemon[839]: accepted connection from pid 1800, user nix-grpc-daemon (trusted) worker1 # [ 164.074608] nix-daemon[839]: reaped child process 1803, status = succeeded lb # [ 164.094674] niks3-server[1130]: 2026/09/22 10:51:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.094712] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.697Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 164.096359] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.697Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 164.118601] niks3-server[1130]: 2026/09/22 10:51:53 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 164.124405] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.727Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.133454] niks3-server[1130]: 2026/09/22 10:51:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 164.142734] niks3-server[1130]: 2026/09/22 10:51:53 INFO Signed narinfos id=56 count=1 lb # [ 164.153069] niks3-server[1130]: 2026/09/22 10:51:53 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 164.163159] niks3-server[1130]: 2026/09/22 10:51:53 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls lb # [ 164.170386] niks3-server[1130]: 2026/09/22 10:51:53 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo lb # [ 164.183059] niks3-server[1130]: 2026/09/22 10:51:53 INFO Completed upload id=56 worker1 # [ 164.174983] nix-grpc-daemon[1563]: time=2026-09-22T10:51:53.777Z level=INFO msg="Upload complete. (488ms)" worker1 # [ 164.182340] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 164.196290] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 164.202805] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 164.207938] nix-daemon[839]: reaped child process 1793, status = succeeded worker1 # [ 164.221716] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 165.062497] nix-daemon[839]: reaped child process 1807, status = succeeded worker1 # [ 165.239739] nix-daemon[839]: accepted connection from pid 1823, user nix-grpc-daemon (trusted) worker1 # [ 165.255467] nix-daemon[839]: reaped child process 1826, status = succeeded lb # [ 165.279774] niks3-server[1130]: 2026/09/22 10:51:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 165.287218] nix-grpc-daemon[1563]: time=2026-09-22T10:51:54.886Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 165.288793] nix-grpc-daemon[1563]: time=2026-09-22T10:51:54.886Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 165.321574] niks3-server[1130]: 2026/09/22 10:51:54 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 165.332421] niks3-server[1130]: 2026/09/22 10:51:54 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 165.336114] nix-grpc-daemon[1563]: time=2026-09-22T10:51:54.938Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.343992] niks3-server[1130]: 2026/09/22 10:51:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 165.349346] niks3-server[1130]: 2026/09/22 10:51:54 INFO Signed narinfos id=57 count=1 lb # [ 165.351684] niks3-server[1130]: 2026/09/22 10:51:54 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 165.362289] niks3-server[1130]: 2026/09/22 10:51:54 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 165.369625] niks3-server[1130]: 2026/09/22 10:51:54 INFO Completed upload id=57 worker1 # [ 165.360931] nix-grpc-daemon[1563]: time=2026-09-22T10:51:54.963Z level=INFO msg="Upload complete. (329ms)" lb # [ 165.372412] niks3-server[1130]: 2026/09/22 10:51:54 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 165.373091] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=1 worker1 # [ 165.387754] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:54Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 165.392513] nix-daemon[839]: reaped child process 1805, status = succeeded worker1 # [ 165.414027] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 165.420874] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:55Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 165.430862] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 165.443519] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 165.456109] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 165.647870] nix-daemon[839]: accepted connection from pid 1832, user nix-grpc-daemon (trusted) worker1 # [ 165.663221] nix-daemon[839]: reaped child process 1835, status = succeeded worker1 # [ 165.840706] nix-daemon[839]: accepted connection from pid 1837, user nix-grpc-daemon (trusted) worker1 # [ 165.855632] nix-daemon[839]: reaped child process 1840, status = succeeded lb # [ 165.877763] niks3-server[1130]: 2026/09/22 10:51:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 165.878600] nix-grpc-daemon[1563]: time=2026-09-22T10:51:55.481Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 165.880190] nix-grpc-daemon[1563]: time=2026-09-22T10:51:55.481Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 165.907948] niks3-server[1130]: 2026/09/22 10:51:55 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 165.919677] niks3-server[1130]: 2026/09/22 10:51:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign worker1 # [ 165.916887] nix-grpc-daemon[1563]: time=2026-09-22T10:51:55.519Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.928822] niks3-server[1130]: 2026/09/22 10:51:55 INFO Signed narinfos id=58 count=1 lb # [ 165.935248] niks3-server[1130]: 2026/09/22 10:51:55 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 165.942712] niks3-server[1130]: 2026/09/22 10:51:55 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 165.949013] niks3-server[1130]: 2026/09/22 10:51:55 INFO Completed upload id=58 worker1 # [ 165.940278] nix-grpc-daemon[1563]: time=2026-09-22T10:51:55.543Z level=INFO msg="Upload complete. (470ms)" lb # [ 165.951555] niks3-server[1130]: 2026/09/22 10:51:55 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 165.948117] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 165.959099] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 165.965377] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 165.969874] nix-daemon[839]: reaped child process 1830, status = succeeded worker1 # [ 165.982797] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 167.421067] nix-daemon[839]: reaped child process 1844, status = succeeded worker1 # [ 167.603370] nix-daemon[839]: accepted connection from pid 1860, user nix-grpc-daemon (trusted) worker1 # [ 167.621030] nix-daemon[839]: reaped child process 1863, status = succeeded lb # [ 167.641890] niks3-server[1130]: 2026/09/22 10:51:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 167.644367] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.246Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 167.645794] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.246Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 167.683277] niks3-server[1130]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 167.693381] niks3-server[1130]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst worker1 # [ 167.693269] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.295Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.704725] niks3-server[1130]: 2026/09/22 10:51:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 167.712700] niks3-server[1130]: 2026/09/22 10:51:57 INFO Signed narinfos id=59 count=1 lb # [ 167.714859] niks3-server[1130]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 167.720325] niks3-server[1130]: 2026/09/22 10:51:57 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 167.726599] niks3-server[1130]: 2026/09/22 10:51:57 INFO Completed upload id=59 lb # [ 167.728973] niks3-server[1130]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker1 # [ 167.718505] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.321Z level=INFO msg="Upload complete. (325ms)" worker1 # [ 167.734357] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=1 worker1 # [ 167.752781] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 167.757206] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 167.762115] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 167.766532] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 167.769816] nix-daemon[839]: reaped child process 1842, status = succeeded worker1 # [ 167.772650] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.374Z level=INFO msg="All 1 paths already cached" worker1 # [ 167.777457] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 167.785504] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 167.968771] nix-daemon[839]: accepted connection from pid 1869, user nix-grpc-daemon (trusted) worker1 # [ 167.985529] nix-daemon[839]: reaped child process 1872, status = succeeded worker1 # [ 168.155578] nix-daemon[839]: accepted connection from pid 1874, user nix-grpc-daemon (trusted) worker1 # [ 168.172852] nix-daemon[839]: reaped child process 1877, status = succeeded lb # [ 168.193662] niks3-server[1130]: 2026/09/22 10:51:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 168.194448] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.797Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 168.195915] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.797Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 168.224424] niks3-server[1130]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 168.222421] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.825Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.233480] niks3-server[1130]: 2026/09/22 10:51:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign lb # [ 168.244653] niks3-server[1130]: 2026/09/22 10:51:57 INFO Signed narinfos id=60 count=1 lb # [ 168.247594] niks3-server[1130]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 168.255604] niks3-server[1130]: 2026/09/22 10:51:57 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 168.262418] niks3-server[1130]: 2026/09/22 10:51:57 INFO Completed upload id=60 worker1 # [ 168.254224] nix-grpc-daemon[1563]: time=2026-09-22T10:51:57.857Z level=INFO msg="Upload complete. (458ms)" lb # [ 168.265058] niks3-server[1130]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 168.260205] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 168.272573] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 168.280278] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 168.283201] nix-daemon[839]: reaped child process 1867, status = succeeded worker1 # [ 168.293614] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 169.134029] nix-daemon[839]: reaped child process 1881, status = succeeded worker1 # [ 169.316727] nix-daemon[839]: accepted connection from pid 1897, user nix-grpc-daemon (trusted) worker1 # [ 169.334433] nix-daemon[839]: reaped child process 1900, status = succeeded lb # [ 169.355274] niks3-server[1130]: 2026/09/22 10:51:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 169.356444] nix-grpc-daemon[1563]: time=2026-09-22T10:51:58.958Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 169.358313] nix-grpc-daemon[1563]: time=2026-09-22T10:51:58.958Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 169.398891] niks3-server[1130]: 2026/09/22 10:51:58 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst worker1 # [ 169.401262] nix-grpc-daemon[1563]: time=2026-09-22T10:51:59.003Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.413913] niks3-server[1130]: 2026/09/22 10:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 169.423066] niks3-server[1130]: 2026/09/22 10:51:58 INFO Signed narinfos id=61 count=1 lb # [ 169.426389] niks3-server[1130]: 2026/09/22 10:51:58 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 169.433815] niks3-server[1130]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 169.437279] niks3-server[1130]: 2026/09/22 10:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 169.443103] niks3-server[1130]: 2026/09/22 10:51:59 INFO Completed upload id=61 worker1 # [ 169.433254] nix-grpc-daemon[1563]: time=2026-09-22T10:51:59.036Z level=INFO msg="Upload complete. (329ms)" lb # [ 169.445274] niks3-server[1130]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 169.450455] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=1 worker1 # [ 169.464403] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 169.476099] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 169.480731] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 169.485458] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 169.488944] nix-daemon[839]: reaped child process 1879, status = succeeded worker1 # [ 169.498610] nix-grpc-daemon[1563]: time=2026-09-22T10:51:59.082Z level=INFO msg="All 1 paths already cached" worker1 # [ 169.505579] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 169.507628] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 169.688463] nix-daemon[839]: accepted connection from pid 1906, user nix-grpc-daemon (trusted) worker1 # [ 169.707344] nix-daemon[839]: reaped child process 1909, status = succeeded worker1 # [ 169.887482] nix-daemon[839]: accepted connection from pid 1911, user nix-grpc-daemon (trusted) worker1 # [ 169.904302] nix-daemon[839]: reaped child process 1914, status = succeeded lb # [ 169.927215] niks3-server[1130]: 2026/09/22 10:51:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 169.928075] nix-grpc-daemon[1563]: time=2026-09-22T10:51:59.530Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 169.929547] nix-grpc-daemon[1563]: time=2026-09-22T10:51:59.530Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 169.956609] niks3-server[1130]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst lb # [ 169.970831] niks3-server[1130]: 2026/09/22 10:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 169.969220] nix-grpc-daemon[1563]: time=2026-09-22T10:51:59.571Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.983341] niks3-server[1130]: 2026/09/22 10:51:59 INFO Signed narinfos id=62 count=1 lb # [ 169.990032] niks3-server[1130]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls worker1 # [ 169.990110] nix-grpc-daemon[1563]: time=2026-09-22T10:51:59.592Z level=INFO msg="Upload complete. (476ms)" lb # [ 169.998856] niks3-server[1130]: 2026/09/22 10:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 170.004369] niks3-server[1130]: 2026/09/22 10:51:59 INFO Completed upload id=62 lb # [ 170.007594] niks3-server[1130]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 170.000155] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 170.010062] nix-grpc-daemon[1486]: ts=2026-09-22T10:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 170.015729] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 170.020396] nix-daemon[839]: reaped child process 1904, status = succeeded worker1 # [ 170.030739] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 170.660580] nix-daemon[839]: reaped child process 1918, status = succeeded worker1 # [ 170.828569] nix-daemon[839]: accepted connection from pid 1934, user nix-grpc-daemon (trusted) worker1 # [ 170.849205] nix-daemon[839]: reaped child process 1937, status = succeeded lb # [ 170.870231] niks3-server[1130]: 2026/09/22 10:52:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 170.872384] nix-grpc-daemon[1563]: time=2026-09-22T10:52:00.474Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 170.873898] nix-grpc-daemon[1563]: time=2026-09-22T10:52:00.474Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 170.897558] niks3-server[1130]: 2026/09/22 10:52:00 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 170.919031] niks3-server[1130]: 2026/09/22 10:52:00 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 170.928384] niks3-server[1130]: 2026/09/22 10:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign worker1 # [ 170.925467] nix-grpc-daemon[1563]: time=2026-09-22T10:52:00.528Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.935696] niks3-server[1130]: 2026/09/22 10:52:00 INFO Signed narinfos id=63 count=1 lb # [ 170.940502] niks3-server[1130]: 2026/09/22 10:52:00 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 170.947542] niks3-server[1130]: 2026/09/22 10:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 170.954106] niks3-server[1130]: 2026/09/22 10:52:00 INFO Completed upload id=63 worker1 # [ 170.945258] nix-grpc-daemon[1563]: time=2026-09-22T10:52:00.548Z level=INFO msg="Upload complete. (304ms)" lb # [ 170.956899] niks3-server[1130]: 2026/09/22 10:52:00 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 170.956878] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=1 worker1 # [ 170.970711] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:00Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 170.974775] nix-daemon[839]: reaped child process 1916, status = succeeded worker1 # [ 171.001468] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 171.011752] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:00Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 171.022022] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 171.035022] nix-grpc-daemon[1563]: time=2026-09-22T10:52:00.637Z level=INFO msg="All 1 paths already cached" worker1 # [ 171.043976] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 171.057377] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 171.248671] nix-daemon[839]: accepted connection from pid 1943, user nix-grpc-daemon (trusted) worker1 # [ 171.267372] nix-daemon[839]: reaped child process 1946, status = succeeded worker1 # [ 171.441649] nix-daemon[839]: accepted connection from pid 1948, user nix-grpc-daemon (trusted) worker1 # [ 171.457854] nix-daemon[839]: reaped child process 1951, status = succeeded lb # [ 171.481857] niks3-server[1130]: 2026/09/22 10:52:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 171.482368] nix-grpc-daemon[1563]: time=2026-09-22T10:52:01.085Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 171.483858] nix-grpc-daemon[1563]: time=2026-09-22T10:52:01.085Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 171.506982] niks3-server[1130]: 2026/09/22 10:52:01 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst lb # [ 171.520843] niks3-server[1130]: 2026/09/22 10:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign worker1 # [ 171.519584] nix-grpc-daemon[1563]: time=2026-09-22T10:52:01.122Z level=INFO msg="Uploading 1 narinfos" lb # [ 171.532425] niks3-server[1130]: 2026/09/22 10:52:01 INFO Signed narinfos id=64 count=1 lb # [ 171.535306] niks3-server[1130]: 2026/09/22 10:52:01 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls lb # [ 171.546906] niks3-server[1130]: 2026/09/22 10:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 171.551967] niks3-server[1130]: 2026/09/22 10:52:01 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo lb # [ 171.556259] niks3-server[1130]: 2026/09/22 10:52:01 INFO Completed upload id=64 worker1 # [ 171.548060] nix-grpc-daemon[1563]: time=2026-09-22T10:52:01.150Z level=INFO msg="Upload complete. (477ms)" worker1 # [ 171.556277] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 171.567448] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 171.573409] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 171.577977] nix-daemon[839]: reaped child process 1941, status = succeeded worker1 # [ 171.590603] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 172.633178] nix-daemon[839]: reaped child process 1955, status = succeeded worker1 # [ 172.810772] nix-daemon[839]: accepted connection from pid 1971, user nix-grpc-daemon (trusted) worker1 # [ 172.826461] nix-daemon[839]: reaped child process 1974, status = succeeded lb # [ 172.850119] niks3-server[1130]: 2026/09/22 10:52:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.852420] nix-grpc-daemon[1563]: time=2026-09-22T10:52:02.452Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 172.853892] nix-grpc-daemon[1563]: time=2026-09-22T10:52:02.452Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 172.889407] niks3-server[1130]: 2026/09/22 10:52:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign worker1 # [ 172.890993] nix-grpc-daemon[1563]: time=2026-09-22T10:52:02.493Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.905977] niks3-server[1130]: 2026/09/22 10:52:02 INFO Signed narinfos id=65 count=1 lb # [ 172.913351] niks3-server[1130]: 2026/09/22 10:52:02 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 172.926719] niks3-server[1130]: 2026/09/22 10:52:02 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 172.931921] niks3-server[1130]: 2026/09/22 10:52:02 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 172.936199] niks3-server[1130]: 2026/09/22 10:52:02 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker1 # [ 172.927921] nix-grpc-daemon[1563]: time=2026-09-22T10:52:02.530Z level=INFO msg="Upload complete. (328ms)" lb # [ 172.939956] niks3-server[1130]: 2026/09/22 10:52:02 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo lb # [ 172.944266] niks3-server[1130]: 2026/09/22 10:52:02 INFO Completed upload id=65 worker1 # [ 172.940467] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=1 worker1 # [ 172.960821] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:02Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.967716] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 172.973315] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:02Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 172.977316] nix-daemon[839]: reaped child process 1953, status = succeeded worker1 # [ 172.995776] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 173.008957] nix-grpc-daemon[1563]: time=2026-09-22T10:52:02.611Z level=INFO msg="All 2 paths already cached" worker1 # [ 173.017197] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 173.029585] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 173.231585] nix-daemon[839]: accepted connection from pid 1980, user nix-grpc-daemon (trusted) worker1 # [ 173.247192] nix-daemon[839]: reaped child process 1983, status = succeeded worker1 # [ 173.427637] nix-daemon[839]: accepted connection from pid 1985, user nix-grpc-daemon (trusted) worker1 # [ 173.441875] nix-daemon[839]: reaped child process 1988, status = succeeded lb # [ 173.465999] niks3-server[1130]: 2026/09/22 10:52:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 173.466295] nix-grpc-daemon[1563]: time=2026-09-22T10:52:03.068Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 173.467780] nix-grpc-daemon[1563]: time=2026-09-22T10:52:03.069Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 173.489829] niks3-server[1130]: 2026/09/22 10:52:03 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 173.494470] nix-grpc-daemon[1563]: time=2026-09-22T10:52:03.097Z level=INFO msg="Uploading 1 narinfos" lb # [ 173.504213] niks3-server[1130]: 2026/09/22 10:52:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 173.514841] niks3-server[1130]: 2026/09/22 10:52:03 INFO Signed narinfos id=66 count=1 lb # [ 173.523859] niks3-server[1130]: 2026/09/22 10:52:03 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 173.521893] nix-grpc-daemon[1563]: time=2026-09-22T10:52:03.124Z level=INFO msg="Upload complete. (477ms)" lb # [ 173.530344] niks3-server[1130]: 2026/09/22 10:52:03 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 173.535274] niks3-server[1130]: 2026/09/22 10:52:03 INFO Completed upload id=66 worker1 # [ 173.528243] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 173.539584] niks3-server[1130]: 2026/09/22 10:52:03 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 173.537965] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 173.542911] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 173.546831] nix-daemon[839]: reaped child process 1978, status = succeeded worker1 # [ 173.558586] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 175.000567] nix-daemon[839]: reaped child process 1992, status = succeeded worker1 # [ 175.181291] nix-daemon[839]: accepted connection from pid 2008, user nix-grpc-daemon (trusted) worker1 # [ 175.197339] nix-daemon[839]: reaped child process 2011, status = succeeded lb # [ 175.219232] niks3-server[1130]: 2026/09/22 10:52:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 175.222566] nix-grpc-daemon[1563]: time=2026-09-22T10:52:04.824Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 175.224080] nix-grpc-daemon[1563]: time=2026-09-22T10:52:04.824Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" worker1 # [ 175.258728] nix-grpc-daemon[1563]: time=2026-09-22T10:52:04.861Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.256238] niks3-server[1130]: 2026/09/22 10:52:04 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 175.270882] niks3-server[1130]: 2026/09/22 10:52:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 175.281122] niks3-server[1130]: 2026/09/22 10:52:04 INFO Signed narinfos id=67 count=1 lb # [ 175.286349] niks3-server[1130]: 2026/09/22 10:52:04 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker1 # [ 175.285975] nix-grpc-daemon[1563]: time=2026-09-22T10:52:04.888Z level=INFO msg="Upload complete. (318ms)" lb # [ 175.295378] niks3-server[1130]: 2026/09/22 10:52:04 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 175.300549] niks3-server[1130]: 2026/09/22 10:52:04 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 175.304540] niks3-server[1130]: 2026/09/22 10:52:04 INFO Completed upload id=67 lb # [ 175.307145] niks3-server[1130]: 2026/09/22 10:52:04 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker1 # [ 175.297434] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=1 worker1 # [ 175.311148] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:04Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 175.315766] nix-daemon[839]: reaped child process 1990, status = succeeded worker1 # [ 175.339771] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 175.354207] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 175.367873] nix-grpc-daemon[1563]: time=2026-09-22T10:52:04.970Z level=INFO msg="All 2 paths already cached" worker1 # [ 175.376591] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 175.389868] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 175.588261] nix-daemon[839]: accepted connection from pid 2015, user nix-grpc-daemon (trusted) worker1 # [ 175.603545] nix-daemon[839]: reaped child process 2018, status = succeeded worker1 # [ 175.779546] nix-daemon[839]: accepted connection from pid 2020, user nix-grpc-daemon (trusted) worker1 # [ 175.794087] nix-daemon[839]: reaped child process 2023, status = succeeded lb # [ 175.818192] niks3-server[1130]: 2026/09/22 10:52:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 175.818301] nix-grpc-daemon[1563]: time=2026-09-22T10:52:05.420Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 175.819766] nix-grpc-daemon[1563]: time=2026-09-22T10:52:05.421Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 175.843162] niks3-server[1130]: 2026/09/22 10:52:05 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 175.848732] nix-grpc-daemon[1563]: time=2026-09-22T10:52:05.451Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.859913] niks3-server[1130]: 2026/09/22 10:52:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 175.870357] niks3-server[1130]: 2026/09/22 10:52:05 INFO Signed narinfos id=68 count=1 lb # [ 175.876209] niks3-server[1130]: 2026/09/22 10:52:05 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 175.884644] niks3-server[1130]: 2026/09/22 10:52:05 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete worker1 # [ 175.880741] nix-grpc-daemon[1563]: time=2026-09-22T10:52:05.483Z level=INFO msg="Upload complete. (475ms)" lb # [ 175.889642] niks3-server[1130]: 2026/09/22 10:52:05 INFO Completed upload id=68 lb # [ 175.893429] niks3-server[1130]: 2026/09/22 10:52:05 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 175.888142] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 175.898862] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 175.904765] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 175.909318] nix-daemon[839]: reaped child process 2013, status = succeeded worker1 # [ 175.921631] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 177.562702] nix-daemon[839]: reaped child process 2027, status = succeeded worker1 # [ 177.748702] nix-daemon[839]: accepted connection from pid 2043, user nix-grpc-daemon (trusted) worker1 # [ 177.765154] nix-daemon[839]: reaped child process 2046, status = succeeded lb # [ 177.790332] niks3-server[1130]: 2026/09/22 10:52:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 177.792431] nix-grpc-daemon[1563]: time=2026-09-22T10:52:07.394Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 177.797268] nix-grpc-daemon[1563]: time=2026-09-22T10:52:07.394Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 177.825466] niks3-server[1130]: 2026/09/22 10:52:07 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 177.837487] niks3-server[1130]: 2026/09/22 10:52:07 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 177.848389] niks3-server[1130]: 2026/09/22 10:52:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign worker1 # [ 177.845460] nix-grpc-daemon[1563]: time=2026-09-22T10:52:07.448Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.857608] niks3-server[1130]: 2026/09/22 10:52:07 INFO Signed narinfos id=69 count=1 lb # [ 177.863120] niks3-server[1130]: 2026/09/22 10:52:07 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 177.871059] niks3-server[1130]: 2026/09/22 10:52:07 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 177.879774] niks3-server[1130]: 2026/09/22 10:52:07 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo lb # [ 177.885489] niks3-server[1130]: 2026/09/22 10:52:07 INFO Completed upload id=69 worker1 # [ 177.877502] nix-grpc-daemon[1563]: time=2026-09-22T10:52:07.480Z level=INFO msg="Upload complete. (344ms)" worker1 # [ 177.893166] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=1 worker1 # [ 177.906684] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:07Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 177.911591] nix-daemon[839]: reaped child process 2025, status = succeeded worker1 # [ 177.934006] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 177.945438] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 177.960281] nix-grpc-daemon[1563]: time=2026-09-22T10:52:07.563Z level=INFO msg="All 2 paths already cached" worker1 # [ 177.969113] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 177.983214] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 178.178965] nix-daemon[839]: accepted connection from pid 2050, user nix-grpc-daemon (trusted) worker1 # [ 178.197112] nix-daemon[839]: reaped child process 2053, status = succeeded worker1 # [ 178.370808] nix-daemon[839]: accepted connection from pid 2055, user nix-grpc-daemon (trusted) worker1 # [ 178.386442] nix-daemon[839]: reaped child process 2058, status = succeeded lb # [ 178.407735] niks3-server[1130]: 2026/09/22 10:52:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 178.408108] nix-grpc-daemon[1563]: time=2026-09-22T10:52:08.010Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 178.409546] nix-grpc-daemon[1563]: time=2026-09-22T10:52:08.010Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 178.435292] niks3-server[1130]: 2026/09/22 10:52:08 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 178.440216] nix-grpc-daemon[1563]: time=2026-09-22T10:52:08.042Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.448783] niks3-server[1130]: 2026/09/22 10:52:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 178.459352] niks3-server[1130]: 2026/09/22 10:52:08 INFO Signed narinfos id=70 count=1 lb # [ 178.465067] niks3-server[1130]: 2026/09/22 10:52:08 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls worker1 # [ 178.464471] nix-grpc-daemon[1563]: time=2026-09-22T10:52:08.067Z level=INFO msg="Upload complete. (464ms)" lb # [ 178.473431] niks3-server[1130]: 2026/09/22 10:52:08 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 178.477939] niks3-server[1130]: 2026/09/22 10:52:08 INFO Completed upload id=70 lb # [ 178.480195] niks3-server[1130]: 2026/09/22 10:52:08 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 178.472116] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 178.482981] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 178.488865] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 178.493100] nix-daemon[839]: reaped child process 2048, status = succeeded worker1 # [ 178.505116] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted) worker1 # [ 179.743375] nix-daemon[839]: reaped child process 2062, status = succeeded worker1 # [ 179.918921] nix-daemon[839]: accepted connection from pid 2078, user nix-grpc-daemon (trusted) worker1 # [ 179.935780] nix-daemon[839]: reaped child process 2081, status = succeeded lb # [ 179.957347] niks3-server[1130]: 2026/09/22 10:52:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 179.960423] nix-grpc-daemon[1563]: time=2026-09-22T10:52:09.562Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 179.962090] nix-grpc-daemon[1563]: time=2026-09-22T10:52:09.562Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 179.996094] niks3-server[1130]: 2026/09/22 10:52:09 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst lb # [ 180.007729] niks3-server[1130]: 2026/09/22 10:52:09 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 180.016300] niks3-server[1130]: 2026/09/22 10:52:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign worker1 # [ 180.014400] nix-grpc-daemon[1563]: time=2026-09-22T10:52:09.616Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.022235] niks3-server[1130]: 2026/09/22 10:52:09 INFO Signed narinfos id=71 count=1 lb # [ 180.026803] niks3-server[1130]: 2026/09/22 10:52:09 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 180.037915] niks3-server[1130]: 2026/09/22 10:52:09 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 180.045073] niks3-server[1130]: 2026/09/22 10:52:09 INFO Completed upload id=71 worker1 # [ 180.036438] nix-grpc-daemon[1563]: time=2026-09-22T10:52:09.639Z level=INFO msg="Upload complete. (321ms)" lb # [ 180.048050] niks3-server[1130]: 2026/09/22 10:52:09 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 180.048841] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37084 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=1 worker1 # [ 180.061977] nix-daemon[839]: reaped child process 2060, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.65 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 180.351899] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 5659e64a170b4dad97b693f2cdb23b9d client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker1 # [ 180.562071] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37084 client # [ 180.603744] nix[1423]: don't know how to build these paths: client # [ 180.604898] nix[1423]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2 # [ 180.582587] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 180.599814] nix-grpc-daemon[1908]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5) worker2 # [ 180.663561] nix-grpc-daemon[1908]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5) worker2 # [ 180.786110] nix-grpc-daemon[1908]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 3/5) worker2 # [ 180.842626] nix-grpc-daemon[1908]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 353 ms (attempt 4/5) worker2 # [ 181.203909] nix-grpc-daemon[1908]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 181.207484] nix-grpc-daemon[1908]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 181.230977] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:10Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 client # [ 181.267546] nix[1423]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker lb: must succeed: curl -sf localhost:9901/clusters lb # [ 181.591736] envoy[927]: conn peer=[2001:db8:1::1]:60016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27200 tx=11445 ms=18331 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code client # [ 211.269094] nix[1423]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu} client # [ 211.278597] nix[1423]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 211.326855] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 211.333940] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 211.337206] systemd[1]: gpu.service: Consumed 178ms CPU time over 30.979s wall clock time, 35.4M memory peak, 5.4K incoming IP traffic, 5.1K outgoing IP traffic. lb # [ 211.589635] envoy[927]: conn peer=[2001:db8:1::1]:57576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=914 tx=540 ms=30695 client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 30.31 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 50.84 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.11 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 213.494529] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:43Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 213.780415] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40662 worker1 # [ 213.774968] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37084 worker1 # [ 213.814560] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:43Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker1 # [ 213.821804] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:43Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 213.869554] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 213.886634] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40662 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 213.904117] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 213.920777] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 213.950972] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:43Z level=info event=niks3_push_started pid=1968 worker2 # [ 214.174942] nix-grpc-daemon[1968]: time=2026-09-22T10:52:43.777Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca="" worker2 # [ 214.358953] nix-daemon[866]: accepted connection from pid 1981, user nix-grpc-daemon (trusted) worker2 # [ 214.377304] nix-daemon[866]: reaped child process 1984, status = succeeded worker2 # [ 214.557577] nix-daemon[866]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 214.576540] nix-daemon[866]: reaped child process 1989, status = succeeded lb # [ 214.593406] niks3-server[1130]: 2026/09/22 10:52:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 214.604075] nix-grpc-daemon[1968]: time=2026-09-22T10:52:44.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 214.606722] nix-grpc-daemon[1968]: time=2026-09-22T10:52:44.206Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 214.630956] niks3-server[1130]: 2026/09/22 10:52:44 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 214.643092] nix-grpc-daemon[1968]: time=2026-09-22T10:52:44.245Z level=INFO msg="Uploading 1 narinfos" lb # [ 214.641980] niks3-server[1130]: 2026/09/22 10:52:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 214.651771] niks3-server[1130]: 2026/09/22 10:52:44 INFO Signed narinfos id=72 count=1 lb # [ 214.661490] niks3-server[1130]: 2026/09/22 10:52:44 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 214.672462] niks3-server[1130]: 2026/09/22 10:52:44 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 214.680781] niks3-server[1130]: 2026/09/22 10:52:44 INFO Completed upload id=72 worker2 # [ 214.680741] nix-grpc-daemon[1968]: time=2026-09-22T10:52:44.283Z level=INFO msg="Upload complete. (501ms)" lb # [ 214.683565] niks3-server[1130]: 2026/09/22 10:52:44 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 214.688150] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40662 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 214.703434] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 214.709947] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 214.714935] nix-daemon[866]: reaped child process 1966, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 214.729842] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 216.169722] nix-daemon[866]: reaped child process 1993, status = succeeded worker2 # [ 216.353144] nix-daemon[866]: accepted connection from pid 2009, user nix-grpc-daemon (trusted) worker2 # [ 216.370180] nix-daemon[866]: reaped child process 2012, status = succeeded lb # [ 216.383863] niks3-server[1130]: 2026/09/22 10:52:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 216.396395] nix-grpc-daemon[1968]: time=2026-09-22T10:52:45.997Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 216.397917] nix-grpc-daemon[1968]: time=2026-09-22T10:52:45.997Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 216.416727] niks3-server[1130]: 2026/09/22 10:52:45 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 216.435821] niks3-server[1130]: 2026/09/22 10:52:46 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 216.446687] niks3-server[1130]: 2026/09/22 10:52:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign worker2 # [ 216.452143] nix-grpc-daemon[1968]: time=2026-09-22T10:52:46.054Z level=INFO msg="Uploading 1 narinfos" lb # [ 216.452696] niks3-server[1130]: 2026/09/22 10:52:46 INFO Signed narinfos id=73 count=1 lb # [ 216.458055] niks3-server[1130]: 2026/09/22 10:52:46 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 216.466593] niks3-server[1130]: 2026/09/22 10:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 216.473415] niks3-server[1130]: 2026/09/22 10:52:46 INFO Completed upload id=73 worker2 # [ 216.473348] nix-grpc-daemon[1968]: time=2026-09-22T10:52:46.076Z level=INFO msg="Upload complete. (332ms)" lb # [ 216.475942] niks3-server[1130]: 2026/09/22 10:52:46 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker1 # [ 216.481201] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:46Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 216.484998] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=1 worker1 # [ 216.486812] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:46Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 216.501143] nix-daemon[866]: reaped child process 1991, status = succeeded worker2 # [ 216.541248] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 216.556391] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 216.569736] nix-grpc-daemon[1968]: time=2026-09-22T10:52:46.172Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 216.579544] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 216.594034] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 216.793240] nix-daemon[866]: accepted connection from pid 2016, user nix-grpc-daemon (trusted) worker2 # [ 216.809998] nix-daemon[866]: reaped child process 2019, status = succeeded worker2 # [ 216.981639] nix-daemon[866]: accepted connection from pid 2021, user nix-grpc-daemon (trusted) worker2 # [ 216.997598] nix-daemon[866]: reaped child process 2024, status = succeeded lb # [ 217.008400] niks3-server[1130]: 2026/09/22 10:52:46 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 217.017616] nix-grpc-daemon[1968]: time=2026-09-22T10:52:46.620Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 217.019097] nix-grpc-daemon[1968]: time=2026-09-22T10:52:46.620Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 217.035548] niks3-server[1130]: 2026/09/22 10:52:46 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst lb # [ 217.047566] niks3-server[1130]: 2026/09/22 10:52:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign worker2 # [ 217.054492] nix-grpc-daemon[1968]: time=2026-09-22T10:52:46.657Z level=INFO msg="Uploading 1 narinfos" lb # [ 217.058301] niks3-server[1130]: 2026/09/22 10:52:46 INFO Signed narinfos id=74 count=1 lb # [ 217.065222] niks3-server[1130]: 2026/09/22 10:52:46 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 217.072691] niks3-server[1130]: 2026/09/22 10:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 217.078483] niks3-server[1130]: 2026/09/22 10:52:46 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo lb # [ 217.083484] niks3-server[1130]: 2026/09/22 10:52:46 INFO Completed upload id=74 worker2 # [ 217.084159] nix-grpc-daemon[1968]: time=2026-09-22T10:52:46.687Z level=INFO msg="Upload complete. (473ms)" worker2 # [ 217.096271] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40662 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 217.105246] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 217.112432] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 217.119136] nix-daemon[866]: reaped child process 2014, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 217.125997] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 218.146910] nix-daemon[866]: reaped child process 2028, status = succeeded worker2 # [ 218.310566] nix-daemon[866]: accepted connection from pid 2044, user nix-grpc-daemon (trusted) worker2 # [ 218.329867] nix-daemon[866]: reaped child process 2047, status = succeeded lb # [ 218.343811] niks3-server[1130]: 2026/09/22 10:52:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 218.356671] nix-grpc-daemon[1968]: time=2026-09-22T10:52:47.958Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 218.358134] nix-grpc-daemon[1968]: time=2026-09-22T10:52:47.959Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 218.382783] niks3-server[1130]: 2026/09/22 10:52:47 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 218.400442] niks3-server[1130]: 2026/09/22 10:52:47 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 218.407952] niks3-server[1130]: 2026/09/22 10:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign worker2 # [ 218.413021] nix-grpc-daemon[1968]: time=2026-09-22T10:52:48.015Z level=INFO msg="Uploading 1 narinfos" lb # [ 218.416985] niks3-server[1130]: 2026/09/22 10:52:47 INFO Signed narinfos id=75 count=1 lb # [ 218.418958] niks3-server[1130]: 2026/09/22 10:52:47 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 218.427225] niks3-server[1130]: 2026/09/22 10:52:48 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 218.433490] niks3-server[1130]: 2026/09/22 10:52:48 INFO Completed upload id=75 lb # [ 218.435789] niks3-server[1130]: 2026/09/22 10:52:48 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 218.434185] nix-grpc-daemon[1968]: time=2026-09-22T10:52:48.037Z level=INFO msg="Upload complete. (300ms)" worker2 # [ 218.445263] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=1 worker1 # [ 218.441269] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:48Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker1 # [ 218.446996] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:48Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 218.461927] nix-daemon[866]: reaped child process 2026, status = succeeded worker2 # [ 218.496991] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 218.512555] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 218.526636] nix-grpc-daemon[1968]: time=2026-09-22T10:52:48.129Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 218.536947] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 218.550692] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 218.760895] nix-daemon[866]: accepted connection from pid 2051, user nix-grpc-daemon (trusted) worker2 # [ 218.781323] nix-daemon[866]: reaped child process 2054, status = succeeded worker2 # [ 218.959481] nix-daemon[866]: accepted connection from pid 2056, user nix-grpc-daemon (trusted) worker2 # [ 218.977389] nix-daemon[866]: reaped child process 2059, status = succeeded lb # [ 218.992560] niks3-server[1130]: 2026/09/22 10:52:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 219.009745] nix-grpc-daemon[1968]: time=2026-09-22T10:52:48.612Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 219.013559] nix-grpc-daemon[1968]: time=2026-09-22T10:52:48.612Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 219.033254] niks3-server[1130]: 2026/09/22 10:52:48 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 219.051043] nix-grpc-daemon[1968]: time=2026-09-22T10:52:48.653Z level=INFO msg="Uploading 1 narinfos" lb # [ 219.045132] niks3-server[1130]: 2026/09/22 10:52:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign lb # [ 219.054963] niks3-server[1130]: 2026/09/22 10:52:48 INFO Signed narinfos id=76 count=1 lb # [ 219.059732] niks3-server[1130]: 2026/09/22 10:52:48 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls lb # [ 219.069332] niks3-server[1130]: 2026/09/22 10:52:48 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 219.073606] niks3-server[1130]: 2026/09/22 10:52:48 INFO Completed upload id=76 worker2 # [ 219.073571] nix-grpc-daemon[1968]: time=2026-09-22T10:52:48.676Z level=INFO msg="Upload complete. (506ms)" lb # [ 219.075690] niks3-server[1130]: 2026/09/22 10:52:48 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 219.080316] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40662 duration_s=0 paths=1 nar_bytes_in=760 worker2 # [ 219.095503] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 worker2 # [ 219.103048] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 219.108196] nix-daemon[866]: reaped child process 2049, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 219.121445] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 220.158349] nix-daemon[866]: reaped child process 2063, status = succeeded worker2 # [ 220.338243] nix-daemon[866]: accepted connection from pid 2079, user nix-grpc-daemon (trusted) worker2 # [ 220.355041] nix-daemon[866]: reaped child process 2082, status = succeeded lb # [ 220.366796] niks3-server[1130]: 2026/09/22 10:52:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 220.380399] nix-grpc-daemon[1968]: time=2026-09-22T10:52:49.982Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 220.381870] nix-grpc-daemon[1968]: time=2026-09-22T10:52:49.982Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 220.407809] niks3-server[1130]: 2026/09/22 10:52:49 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 220.418996] niks3-server[1130]: 2026/09/22 10:52:49 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst lb # [ 220.427951] niks3-server[1130]: 2026/09/22 10:52:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker2 # [ 220.433705] nix-grpc-daemon[1968]: time=2026-09-22T10:52:50.036Z level=INFO msg="Uploading 1 narinfos" lb # [ 220.436215] niks3-server[1130]: 2026/09/22 10:52:50 INFO Signed narinfos id=77 count=1 lb # [ 220.438775] niks3-server[1130]: 2026/09/22 10:52:50 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 220.452385] niks3-server[1130]: 2026/09/22 10:52:50 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 220.457445] niks3-server[1130]: 2026/09/22 10:52:50 INFO Completed upload id=77 worker2 # [ 220.457705] nix-grpc-daemon[1968]: time=2026-09-22T10:52:50.060Z level=INFO msg="Upload complete. (326ms)" lb # [ 220.460552] niks3-server[1130]: 2026/09/22 10:52:50 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 220.469100] nix-grpc-daemon[1908]: ts=2026-09-22T10:52:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40662 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=1 worker2 # [ 220.486491] nix-daemon[866]: reaped child process 2061, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.00 seconds) client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 221.498187] nix-grpc-daemon[1486]: ts=2026-09-22T10:52:51Z level=info event=healthy reason=min_free lb # [ 221.588827] envoy[927]: conn peer=[2001:db8:1::1]:45466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9575 tx=4141 ms=6704 worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.76 seconds) (finished: run the VM test script, in 223.08 seconds) test script finished in 223.37s cleanup kill QemuMachine (pid 50) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) kill QemuMachine (pid 49) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) kill QemuMachine (pid 52) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) kill QemuMachine (pid 51) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14) (finished: cleanup, in 1.15 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh